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Abstract The core of a formal semantics of an imperative programming language is 
a memory model that describes the behavior of operations on the memory. Dehning a 
memory model that matches the description of C in the Cll standard is challenging 
because C allows both high-level (by means of typed expressions) and low-level (by 
means of bit manipulation) memory accesses. The Cll standard has restricted the 
interaction between these two levels to make more effective compiler optimizations 
possible, on the expense of making the memory model complicated. 

We describe a formal memory model of the (non-concurrent part of the) Cll stan¬ 
dard that incorporates these restrictions, and at the same time describes low-level 
memory operations. This formal memory model includes a rich permission model to 
make it usable in separation logic and supports reasoning about program transfor¬ 
mations. The memory model and essential properties of it have been fully formalized 
using the Coq proof assistant. 

Keywords ISO Cll Standard • C Verihcation ■ Memory Models • Separation 
Logic • Interactive Theorem Proving ■ Coq 


1 Introduction 

A memory model is the core of a semantics of an imperative programming language. 
It models the memory states and describes the behavior of memory operations. The 
main operations described by a C memory model are: 

— Reading a value at a given address. 

— Storing a value at a given address. 

— Allocating a new object to hold a local variable or storage obtained via malloc. 

— Deallocating a previously allocated object. 
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Formalizing the Cll memory model in a faithful way is challenging because C 
features both low-level and high-level data access. Low-level data access involves un¬ 
structured and untyped byte representations whereas high-level data access involves 
typed abstract values such as arrays, structs and unions. 

This duality makes the memory model of C more complicated than the memory 
model of nearly any other programming language. For example, more mathematically 
oriented languages such as Java and ML feature only high-level data access, in which 
case the memory can be modeled in a relatively simple and structured way, whereas 
assembly languages feature only low-level data access, in which case the memory can 
be modeled as an array of bits. 

The situation becomes more complicated as the Cll standard allows compilers 
to perform optimizations based on a high-level view of data access that are inconsis¬ 
tent with the traditional low-level view of data access. This complication has lead to 
numerous ambiguities in the standard text related to aliasing, uninitialized memory, 
end-of-array pointers and type-punning that cause problems for C code when com¬ 
piled with widely used compilers. See for example the message [12] on the standard 
committee’s mailing list. Defect Reports #236, #260, and #451 [26], and the various 
examples in this paper. 

Contribution. This paper describes the CH 2 O memory model, which is part of the 
the CH 2 O project |351l36ll30ll3ni3lll32l[371[33| . CH 2 O provides an operational, exe¬ 
cutable and axiomatic semantics in Coq for a large part of the non-concurrent frag¬ 
ment of C, based on the official description of C as given by the Cll standard m- 
The key features of the CH 2 O memory model are as follows: 

— Close to Cll. CH 2 O is faithful to the Cll standard in order to be compiler 
independent. When one proves something about a given program with respect 
to CH 2 O, it should behave that way with any Cll compliant compiler (possibly 
restricted to certain implementation defined choices). 

— Static type system. Given that C is a statically typed language, CH 2 O does 
not only capture the dynamic semantics of Cll but also its type system. We have 
established properties such as type preservation of the memory operations. 

— Proof infrastructure. All parts of the CH 2 O memory model and semantics have 
been formalized in Coq (without axioms). This is essential for its application to 
program verification in proof assistants. Also, considering the significant size of 
CH 2 O and its memory model, proving metatheoretical properties of the language 
would have been intractable without the support of a proof assistant. 

Despite our choice of using Coq, we believe that nearly all parts of CH 2 O could 
be formalized in any proof assistant based on higher-order logic. 

— Executable. To obtain more confidence in the accuracy of CH2O with respect 
to Cll, the CH 2 O memory model is executable. An executable memory model 
allows us to test the CH 2 O semantics on example programs and to compare the 
behavior with that of widely used compilers |3I||33|. 

— Separation logic. In order to reason about concrete C programs, one needs a 
program logic. To that end, the CH 2 O memory model incorporates a complex 
permission model suitable for separation logic. This permission system, as well 
as the memory model itself, forms a separation algebra. 

— Memory refiuemeuts. CH2O has an expressive notion of memory refinements 
that relates memory states. All memory operations are proven invariant under 
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this notion. Memory refinements form a general way to validate many common- 
sense properties of the memory model in a formal way. They also open the door 
to reasoning about program transformations, which is useful if one were to use 
the memory model as part of a verified compiler front-end. 

This paper is an extended version of previously published conference papers at 
CPP [30] and VSTTE [32 ■ In the time following these two publications, the mem¬ 
ory model has been extended significantly and been integrated into an operational, 
executable and axiomatic semantics [371[S3]. The memory model now supports more 
features, various improvements to the definitions have been carried out, and more 
properties have been formally proven as part of the Coq development. 

Parts of this paper also appear in the author’s PhD thesis m , which describes the 
entire CH 2 O project including its operational, executable and axiomatic semantics, 
and metatheoretical results about these. 


Problem. The Cll standard gives compilers a lot of freedom in what behaviors a 
program may have [IZl 3.4]. It uses the following notions of under-specification: 

— Unspecified behavior: two or more behaviors are allowed. For example: the exe¬ 
cution order of expressions. The choice may vary for each use of the construct. 

— Implementation defined behavior: like unspecified behavior, but the compiler has 
to document its choice. For example: size and endianness of integers. 

— Undefined behavior: the standard imposes no requirements at all, the program 
is even allowed to crash. For example: dereferencing a NULL pointer, or signed 
integer overflow. 

Under-specification is used extensively to make C portable, and to allow compilers 
to generate fast code. For example, when dereferencing a pointer, no code has to be 
generated to check whether the pointer is valid or not. If the pointer is invalid (NULL 
or a dangling pointer), the compiled program may do something arbitrary instead of 
having to exit with a NullPointerException as in Java. Since the CH 2 O semantics 
intends to be a formal version of the Cll standard, it has to capture the behavior of 
any C compiler, and thus has to take all under-specification seriously (even if that 
makes the semantics complex). 

Modeling under-specification in a formal semantics is folklore: unspecified behav¬ 
ior corresponds to non-determinism, implementation defined behavior corresponds to 
parametrization, and undefined behavior corresponds to a program having no seman¬ 
tics. However, the extensive amount of underspecification in the Cll standard [m 
Annex J], and especially that with respect to the memory model, makes the situation 
challenging. We will give a motivating example of subtle underspecification in the 
introduction of this paper. Section [3| provides a more extensive overview. 


Motivating example. A drawback for efficient compilation of programming languages 
with pointers is aliasing. Aliasing describes a situation in which multiple pointers 
refer to the same object. In the following example the pointers p and q are said to 
be aliased. 

int x; int *p = &x, *q = &x; 





4 


Robbert Krebbers 


The problem of aliased pointers is that writes through one pointer may effect the 
result of reading through the other pointer. The presence of aliased pointers therefore 
often disallows one to change the order of instructions. For example, consider: 

int f(int *p, int *q) { 

int z = *q; *p = 10; return z; 

} 

When f is called with pointers p and q that are aliased, the assignment to *p 
also affects *q. As a result, one cannot transform the function body of f into the 
shorter *p = 10; return (*q) ;. The shorter function will return 10 in case p and 
q are aliased, whereas the original f will always return the original value of *q. 

Unlike this example, there are many situations in which pointers can be assumed 
not to alias. It is essential for an optimizing compiler to determine where aliasing 
cannot occur, and use this information to generate faster code. The technique of 
determining whether pointers can alias or not is called alias analysis. 

In type-based alias analysis, type information is used to determine whether point¬ 
ers can alias or not. Consider the following example: 

short g(int *p, short *q) { 

short z = *q; *p = 10; return z; 

} 

Here, a compiler is allowed to assume that p and q are not aliased because they 
point to objects of different types. The compiler is therefore allowed to transform 
the function body of g into the shorter *p = lO; return (*q);. 

The peculiar thing is that the C type system does not statically enforce the 
property that pointers to objects of different types are not aliased. A union type can 
be used to create aliased pointers to different types: 


union int_or_short { int x; short y; ]-u = { .y = 3]-; 

int *p = &u.x; 

// p points to 

the X variant of u 

short *q = &u.y; 

// q points to 

the y variant of u 

return g(p, q) ; 

// g is called 

with aliased pointers p and q 


The above program is valid according to the rules of the Cll type system, but has 
undefined behavior during execution of g. This is caused by the standard’s notion 
of effective types mi 6 .5p6-7] (also called strict-aliasing restrictions) that assigns 
undefined behavior to incorrect usage of aliased pointers to different types. 

We will inline part of the function body of g to indicate the incorrect usage of 
aliased pointers during the execution of the example. 

union int_or_short { int x; short y; ]-u = { .y = 3]-; 
int *p = &u.x; // p points to the x variant of u 

short *q = &u.y; // q points to the y variant of u 

// g(p, q) is called, the body is inlined 

short z = *q; // u has variant y and is accessed through y -> OK 

*p = 10; // u has variant y and is accessed through x -> bad 

The assignment *p = 10 violates the rules for effective types. The memory area 
where p points to contains a union whose variant is y of type short, but is accessed 
through a pointer to variant x of type int. This causes undehned behavior. 
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Effective types form a clear tension between the low-level and high-level way of 
data access in C. The low-level representation of the memory is inherently untyped 
and unstructured and therefore does not contain any information about variants of 
unions. However, the standard treats the memory as if it were typed. 

Approach. Most existing C formalizations (most notably Norrish [i^, Leroy et al. |391 
sg and Ellison and Ro§u [19]) use an unstructured untyped memory model where 
each object in the formal memory model consists of an array of bytes. These for¬ 
malizations therefore cannot assign undefined behavior to violations of the rules for 
effective types, among other things. 

In order to formalize the interaction between low-level and high-level data access, 
and in particular effective types, we represent the formal memory state as a forest 
of well-typed trees whose structure corresponds to the structure of data types in C. 
The leaves of these trees consist of bits to capture low-level aspects of the language. 
The key concepts of our memory model are as follows: 

— Memory trees fSection 16.311 are used to represent each object in memory. They 
are abstract trees whose structure corresponds to the shape of C data types. The 
memory tree of struct S { short x, *r; } s = { 33, &s.x } might be (the 
precise shape and the bit representations are implementation defined): 


structs 



The leaves of memory trees contain permission annotated bits (Section |6.2D . Bits 
are represented symbolically: the integer value 33 is represented as its binary 
representation 1000010000000000, the padding bytes as symbolic indeterminate 
bits } (whose actual value should not be used), and the pointer &s. x as a sequence 
of symbolic pointer hits. 

The memory itself is a forest of memory trees. Memory trees are explicit about 
type information (in particular the variants of unions) and thus give rise to a 
natural formalization of effective types. 

— Pointers fSection 16.111 are formalized using paths through memory trees. Since 
we represent pointers as paths, the formal representation contains detailed infor¬ 
mation about how each pointer has been obtained (in particular which variants 
of unions were used). A detailed formal representation of pointers is essential to 
describe effective types. 

— Abstract values (Definition 16.411 are trees whose structure is similar to memory 
trees, but have base values (mathematical integers and pointers) on their leaves. 
The abstract value of struct S { short x, *r; } s = { 33, &s.x }is: 


structs 



Abstract values hide internal details of the memory such as permissions, padding 
and object representations. They are therefore used in the external interface of 
the memory model and throughout the operational semantics. 
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Memory trees, abstract values and bits with permissions can be converted into 
each other. These conversions are used to dehne operations internal to the memory 
model. However, none of these conversions are bijective because different information 
is materialized in these three data types: 



Abstract values 

Memory trees 

Bits with permissions 

Permissions 


y 

y 

Padding 


always / 

y 

Variants of union 

y 

y 


Mathematical values 

y 




This table indicates that abstract values and sequences of bits are complementary. 
Memory trees are a middle ground, and therefore suitable to describe both the low- 
level and high-level aspects of the C memory. 


Outline. This work presents an executable mathematically precise version of a large 
part of the (non-concurrent) C memory model. 

— Section [3] describes some challenges that a Cll memory model should address; 
these include end-of-array pointers, byte-level operations, indeterminate memory, 
and type-punning. 

— Section 3] describes the types of C. Our formal development is parametrized by 
an abstract interface to characterize implementation dehned behavior. 

— Section [3] describes the permission model using a variant of separation algebras 
that is suitable for formalization in Coq. The permission model is built compo- 
sitionally from simple separation algebras. 

— Section [6] contains the main part of this paper, it describes a memory model that 
can accurately deal with the challenges posed in Section [31 

— Section [7] demonstrates that our memory model is suitable for formal proofs. 
We prove that the standard’s notion of effective types has the desired effect of 
allowing type-based alias analysis (Section 17.HI . we present a method to reason 
compositionally about memory transformations (Section l7.2L and prove that the 
memory model has a separation algebra structure (Section 17.41) . 

— Section [8] describes the Coq formalization: all proofs about our memory model 
have been fully formalized using Coq. 

As this paper describes a large formalization effort, we often just give representative 
parts of definitions. The interested reader can find all details online as part of our 
Coq formalization at: 

http://robbertkrebbers.nl/research/ch2o/ 


2 Notations 

This section introduces some common mathematical notions and notations that will 
be used throughout this paper. 

Definition 2.1 We let N denote the type of natural numbers (including 0), let Z 
denote the type of integers, and let Q denote the type of rational numbers. We let 
i I j denote that f £ N is a divisor of j £ N. 
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Definition 2.2 We let Prop denote the type of propositions, and let bool denote the 
type of Booleans whose elements are true and false. Most propositions we consider 
have a corresponding Boolean-valud decision function. In Coq we use type classes to 
keep track of these correspondences, but in this paper we leave these correspondences 
implicit. 

Definition 2.3 We let option A denote the option type over A, whose elements are 
inductively dehned as either ± or a; for some x ^ A. We implicitly lift operations to 
operate on the option type, and often omit cases of definitions that yield _L. This is 
formally described using the option monad in the Coq formalization. 

Definition 2.4 A partial function f from A to B is a function f : A ^ option B. 

Definition 2.5 A partial function / is called a finite partial function or a finite map 
if its domain dom f := {x \ 3y ^ B. f x = y} is hnite. The type of finite partial 
functions is denoted as A — B. The operation f[x := y\ yields / with the value y 
for argument x. 

Definition 2.6 We let A x i? denote the product of types A and B. Given a pair 
{x, y) e A X B, we let {x, y)^ := x and (x, y)^ '■= y denote the first and second 
projection of {x, y). 

Definition 2.7 We let list A denote the list type over A, whose elements are induc¬ 
tively defined as either s or x x for some x € A and x E list A. We let Xi E A denote 
the zth element of a list x E list A (we count from 0). Lists are sometimes denoted 
as [a:o,... ,Xn-i ] E list A for Xq, ... ,x„-i E A. 

We use the following operations on lists: 

— We often implicitly lift a function / : Aq —> • • • ^ A„ point-wise to the function 
/ : list Aq —> ■ ■ ■ —> list A„. The resulting list is truncated to the length of the 
smallest input list in case n > 1. 

— We often implicitly lift a predicate P : Aq ^ A„_i —> Prop to the predicate 
P : list Aq —> • • • —> list A„_i Prop that guarantees that P holds for all (pairs 
of) elements of the list(s). The lifted predicate requires all lists to have the same 
length in case n > 1. 

— We let |a;| E N denote the length of a; € list A. 

— We let X[ij) E list A denote the sublist Xi... Xj-i of x E list A. 

— We let a;" E list A denote the list consisting of n times a; e A. 

— We let (af7/°°)jj € list A denote the sublist Xi.. -Xj-i of x E list A which is 

padded with y E A in case x is too short. 

— Given lists x E list A and y E list B with |a:| = |y|, we let xy E list (A x B) denote 
the point-wise pairing of x and y. 


3 Challenges 

This section illustrates a number of subtle forms of underspecification in C by means 
of example programs, their bizarre behaviors exhibited by widely used G compilers, 
and their treatment in CH 2 O. Many of these examples involve delicacies due to the 
interaction between the following two ways of accessing data: 

— In a high-level way using arrays, structs and unions. 
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— In a low-level way using unstructured and untyped byte representations. 

The main problem is that compilers use a high-level view of data access to perform 
optimizations whereas both programmers and traditional memory models expect 
data access to behave in a concrete low-level way. 


3.1 Byte-level operations and object representations 


Apart from high-level access to objects in memory by means of typed expressions, 
C also allows low-level access by means of byte-wise manipulation. Each object of 
type T can be interpreted as an unsigned char array of length sizeof (r), which is 
called the objeet representation [271 6.2.6.1p4]. Let us consider: 

struct S { short x; short *r; } si = { 10, fesl.x }; 
unsigned char *p = (unsigned char*)&sl; 

On 32-bits computing architectures such as x86 (with _Alignof (short*) = 4), 
the object representation of si might be: 


X padding r 





01010000 

00000000 

uuuu 

HUUii 

•••••••• 

•••••••• 

•••••••• 

•••••••• 


t t T 

P P + 1 P 2 


The above object representation contains a hole due to alignment of objects. The 
bytes belonging such holes are called padding bytes. 

Alignment is the way objects are arranged in memory. In modern computing 
architectures, accesses to addresses that are a multiple of a word sized chunk (often a 
multiple of 4 bytes on a 32-bits computing architecture) are signihcantly faster due to 
the way the processor interacts with the memory. For that reason, the Cll standard 
has put restrictions on the addresses at which objects may be allocated [271 6.2.8]. 
For each type r, there is an implementation defined integer constant _Alignof (t) , 
and objects of type t are required to be allocated at addresses that are a multiple of 
that constant. In case _Alignof (short*) = 4, there are thus two bytes of padding 
in between the helds of struct S. 

An object can be copied by copying its object representation. For example, the 
struct si can be copied to s2 as follows: 

struct S { short x; short *r; } si = { 10, fesl.x }; 
struct S s2; 

for (size_t i = 0; i < sizeof(struct S); i++) 

((unsigned char*)&s2)[i] = ((unsigned char*)&sl) [i]; 

In the above code, size_t is an unsigned integer type, which is able to hold the 
results of the sizeof operator [271 7.19p2]. 

Manipulation of object representations of structs also involves access to padding 
bytes, which are not part of the high-level representation. In particular, in the exam¬ 
ple the padding bytes are also being copied. The problematic part is that padding 
bytes have indeterminate values, whereas in general, reading an indeterminate value 
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has undefined behavior (for example, reading from an uninitialized int variable is un¬ 
defined). The Cll standard provides an exception for unsigned char ^ 7 \ 6.2.6.1p5], 
and the above example thus has defined behavior. 

Our memory model uses a symbolic representation of bits (Definition 16.191) to 
distinguish determinate and indeterminate memory. This way, we can precisely keep 
track of the situations in which access to indeterminate memory is permitted. 


3.2 Padding of structs and unions 

The following excerpt from the Cll standard points out another challenge with 
respect to padding bytes [271 6.2.6.1p6]: 

When a value is stored in an object of structure or union type, including in 
a member object, the bytes of the object representation that correspond to 
any padding bytes take unspecified values. 

Let us illustrate this difficulty by an example: 

struct S { char x; char y; char z; ]■; 

void f (struct S *p) { p->x = 0; p->y = 0; p->z =0; } 

On architectures with sizeof (struct S) = 4, objects of type struct S have 
one byte of padding. The object representation may be as follows: 


X y z padding 



P 


Instead of compiling the function f to three store instructions for each field of 
the struct, the Cll standard allows a compiler to use a single instruction to store 
zeros to the entire struct. This will of course affect the padding byte. Consider: 

struct Ss={l, 1, 1}; 

((unsigned char*)&s) [3] = 10; 
f(&s); 

return ((unsigned char*)&:s) [3] ; 

Now, the assignments to fields of s by the function f affect also the padding bytes 
of s, including the one ((unsigned char*)&s) [3] that we have assigned to. As a 
consequence, the returned value is unspecified. 

From a high-level perspective this behavior makes sense. Padding bytes are not 
part of the abstract value of a struct, so their actual value should not matter. How¬ 
ever, from a low-level perspective it is peculiar. An assignment to a specific field of 
a struct affects the object representation of parts not assigned to. 

None of the currently existing C formalizations describes this behavior correctly. 
In our tree based memory model we enforce that padding bytes always have an 
indeterminate value, and in turn we have the desired behavior implicitly. Note that 
if the function call f (&s) would have been removed, the behavior of the example 
program remains unchanged in CH 2 O. 
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3.3 Type-punning 

Despite the rules for effective types, it is under certain conditions nonetheless allowed 
to access a union through another variant than the current one. Accessing a union 
through another variant is called type-punning. For example: 

union int_or_short { int x; short y; l-u = { .x = 3]-; 
printf ("°/.d\n" , u.y); 

This code will reinterpret the bit representation of the int value 3 of u.x as 
a value of type short. The reinterpreted value that is printed is implementation 
defined (on architectures where shorts do not have trap values). 

Since Cll is ambiguous about the exact conditions under which type-punning is 
allowec(3, we follow the interpretation by the GCC documentation |2U| : 

Type-punning is allowed, provided the memory is accessed through the union 

type. 

According to this interpretation the above program indeed has implementation 
defined behavior because the variant y is accessed via the expression u. y that involves 
the variable u of the corresponding union type. 

However, according to this interpretation, type-punning via a pointer to a specific 
variant of a union type yields undefined behavior. This is in agreement with the rules 
for effective types. For example, the following program has undefined behavior. 

union int_or_short { int x; short y; l-u = { .x = 3]-; 
short *p = &u.y; 
printf ("°/.d\n" , *p) ; 

We formalize the interpretation of Cll by GCC by decorating pointers and 1- 
values to subobjects with annotations lDefinition l6.4D . When a pointer to a variant of 
a union is stored in memory, or used as the argument of a function, the annotations 
are changed to ensure that type-punning no longer has defined behavior via that 
pointer. In Section [7711 we formally establish that this approach is correct by showing 
that a compiler can perform type-based alias analysis (Theorem 17.21 on page [ST]). 


3.4 Indeterminate memory and pointers 

A pointer value becomes indeterminate when the object it points to has reached the 
end of its lifetime [271 6.2.4] (it has gone out of scope, or has been deallocated). 
Dereferencing an indeterminate pointer has of course undefined behavior because it 
no longer points to an actual value. However, not many people are aware that using 
an indeterminate pointer in pointer arithmetic and pointer comparisons also yields 
undefined behavior. Consider: 

int *p = malloc(sizeof(int)); assert (p != NULL); 
free(p); 


^ The term type-punning merely appears in a footnote [27] footnote 95]. There is however the 
related common initial sequence rule 1271 6.5.2.3], for which the Cll standard uses the notion 
of visible. This notion is not clearly defined either. 
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int *q = malloc(sizeof(int)); assert (q != NULL); 
if (p == q) { // undefined, p is indeterminate due to the free 
*q = 10; 

*p = 14; 

printf ("%d\ii", *q) ; // p and q alias, expected to print If. 

} 

In this code malloc (sizeof (int)) yields a pointer to a newly allocated memory 
area that may hold an integer, or yields a NULL pointer in case no memory is available. 
The function free deallocates memory allocated by malloc. In the example we assert 
that both calls to malloc succeed. 

After execution of the second call to malloc it may happen that the memory 
area of the first call to malloc is reused: we have used free to deallocate it after all. 
This would lead to the following situation in memory: 


P result of malloc 1 



Both GCC (version 4.9.2) or Clang (version 3.5.0) use the fact that p and q are 
obtained via different calls to malloc as a license to assume that p and q do not 
alias. As a result, the value 10 of *q is inlined, and the program prints the value 10 
instead of the naively expected value 14. 

The situation becomes more subtle because when the object a pointer points to 
has been deallocated, not just the argument of free becomes indeterminate, but 
also all other copies of that pointer. This is therefore yet another example where 
high-level representations interact subtly with their low-level counterparts. 

In our memory model we represent pointer values symbolically (Definition 16.411 . 
and keep track of memory areas that have been previously deallocated. The behavior 
of operations like == depends on the memory state, which allows us to accurately 
capture the described undefined behaviors. 


3.5 End-of-array pointers 

The way the Cll standard deals with pointer equality is subtle. Consider the follow¬ 
ing excerpt [271 6.5.9p6]: 

Two pointers compare equal if and only if [...] or one is a pointer to one past 
the end of one array object and the other is a pointer to the start of a different 
array object that happens to immediately follow the first array object in the 
address space. 

End-of-array pointers are peculiar because they cannot be dereferenced, they do 
not point to any value after all. Nonetheless, end-of-array are commonly used when 
looping through arrays. 

int a[4] ={0, 1, 2, 3}; 
int *p = a; 

while (p < a + 4) { *p += 1; p += 1; I 
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The pointer p initially refers to the hrst element of the array a. The value p 
points to, as well as p itself, is being increased as long as p is before the end-of-array 
pointer a + 4. This code thus increases the values of the array a. The initial state 
of the memory is displayed below: 


a 


0 

1 

2 

_s 

3 



P a + 4 

End-of-array pointers can also be used in a way where the result of a comparison 
is not well-dehned. In the example below, the printf is executed only if x and y are 
allocated adjacently in the address space (typically the stack). 

int X, y; 

if (&x + 1 == &y) printf("x and y are allocated adjacently\n"); 

Based on the aforementioned excerpt of the Cll standard |271 6.5.9p6], one would 
naively say that the value of &x + 1 == &y is uniquely determined by the way x and 
y are allocated in the address space. However, the GCC implementers disagre^l. 
They claim that Defect Report ^260 [26] allows them to take the derivation of a 
pointer value into account. 

In the example, the pointers &x + 1 and &y are derived from unrelated objects 
(the local variables x and y). As a result, the GCC developers claim that &x + 1 
and fey may compare unequal albeit being allocated adjacently. Consider: 

int compare(int *p, int *q) { 

// some code to confuse the optimizer 
return p == q; 

} 

int mainO { 
int x, y; 

if (&x + 1 == fey) printf ("x eind y are adjacent\n") ; 
if (compare(&x + 1, &y)) printf ("x eind y are still adjacent\n") ; 

} 

When compiled with GCC (version 4.9.2), we have observed that the string x 
and y are still adjacent is being printed, wheras x and y are adjacent is not 
being printed. This means that the value of &x + 1 == &y is not consistent among 
different occurrences of the comparison. 

Due to these discrepancies we assign undehned behavior to questionable uses of 
end-of-array pointers while assigning the correct defined behavior to pointer com¬ 
parisons involving end-of-array pointers when looping through arrays (such as in the 
hrst example above). Our treatment is similar to our extension of CompCert |34| . 


3.6 Sequence point violations and non-determinism 

Instead of having to follow a specihc execution order, the execution order of expres¬ 
sions is unspecihed in C. This is a common cause of portability problems because 


2 


See https://gcc.gnu.org/bugzilla/show_bug.cgi?id=61502 
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a compiler may use an arbitrary execution order for each expression, and each time 
that expression is executed. Hence, to ensure correctness of a C program with respect 
to an arbitrary compiler, one has to verify that each possible execution order is free 
of undefined behavior and gives the correct result. 

In order to make more effective optimizations possible (for example, delaying of 
side-effects and interleaving), the C standard does not allow an object to be modihed 
more than once during the execution of an expression. If an object is modihed more 
than once, the program has undehned behavior. We call this requirement the sequence 
point restriction. Note that this is not a static restriction, but a restriction on valid 
executions of the program. Let us consider an example: 

int x, y = (x = 3) + (x = 4); 
printfCy.d °/.d\n", x, y) ; 

By eonsidering all possible execution orders, one would naively expect this pro¬ 
gram to print 4 7 or 3 7, depending on whether the assignment x = 3 or x = 4 is 
executed hrst. However, x is modihed twice within the same expression, and thus 
both execution orders have undehned behavior. The program is thereby allowed to 
exhibit any behavior. Indeed, when compiled with gcc -02 (version 4.9.2), the com¬ 
piled program prints 4 8, which does not correspond to any of the execution orders. 

Our approach to non-determinism and sequence points is inspired by Norrish [31] 
and Ellison and Ro§u m- Each bit in memory carries a permission (Dehnition IS.SD 
that is set to a special locked permission when a store has been performed. The mem¬ 
ory model prohibits any access (read or store) to objects with locked permissions. 
At the next sequence point, the permissions of locked objects are changed back into 
their original permission, making future accesses possible again. 

It is important to note that we do not have non-determinism in the memory model 
itself, and have set up the memory model in such a way that all non-determinism is 
on the level of the small-step operational semantics. 


4 Types in C 

This section describes the types used in the CH 2 O memory model. We support inte¬ 
ger, pointer, function pointer, array, struct, union and void types. More complicated 
types such as enum types and typedefs are dehned by translation ISIllSSj. 

This section furthermore describes an abstract interface, called an implementa¬ 
tion environment, that describes properties such as size and endianness of integers, 
and the layout of structs and unions. The entire CH 2 O memory model and semantics 
will be parametrized by an implementation environment. 


4.1 Integer representations 

This section describes the part of implementation environments corresponding to 
integer types and the encoding of integer values as bits. Integer types consist of a 
rank (char, short, int ...) and a signedness (signed or unsigned). The set of available 
ranks as well as many of their properties are implementation dehned. We therefore 
abstract over the ranks in the dehnition of integer types. 





14 


Robber! Krebbers 


Definition 4.1 Integer signedness and integer types over ranks k (z K are induc¬ 
tively defined as: 


si e signedness ::= signed | unsigned 
T\ 6 inttype ::= si k 

The projections are called rank : inttype —^ K and sign : inttype —> signedness. 

Definition 4.2 An integer coding environment with ranks K consists of a total order 
{K, C) of integer ranks having at least the following ranks: 

char C short C int C long C long long and ptr_rank. 

ft moreover has the following functions: 

char_bits : N>8 endianize : K —> list bool —> list bool 

char_signedness : signedness deendianize : K —> list bool —> list bool 

rank_size : K N>o 


Here, endianize k and deendianize k should be inverses, endianize k should be a 
permutation, rank_size should be (non-strictly) monotone, and rank_size char = 1. 


Definition 4.3 The judgment x : t\ describes that x has integer type t\. 


2 char_bits*rank_size k — 1 <C k — 1 


Q ^ ^ ^'ts*rank_size k 


X : signed k 


X : unsigned k 


The rank char is the rank of the smallest integer type, whose unsigned variant 
corresponds to bytes that constitute object representations (see Section lOT) . Its bit 
size is char_bits (called CHAR_BIT in the standard library header files |271 5. 2. 4. 2.1]), 
and its signedness char_signedness is implementation defined [271 6.2.5pl5]. 

The rank ptr_rank is the rank of the integer types size_t and ptrdiff_t, which 
are defined in the standard library header files |271 7.19p2]. The type ptrdiff_t is 
a signed integer type used to represent the result of subtracting two pointers, and 
the type size_t is an unsigned integer type used to represent sizes of types. 

An integer coding environment can have an arbitrary number of integer ranks 
apart from the standard ones char, short, int, long, long long, and ptr_rank. This way, 
additional integer types like those describe in |271 7.20] can easily be included. 

The function rank_size gives the byte size of an integer of a given rank. Since 
we require rank_size to be monotone rather than strictly monotone, integer types 
with different ranks can have the same size [23 6.3.1.1pl]. For example, on many 
implementations int and long have the same size, but are in fact different. 

The Cll standard allows implementations to use either sign-magnitude, I’s com¬ 
plement or 2’s complement signed integers representations, ft moreover allows integer 
representations to contain padding or parity bits [271 6.2.6.2]. However, since all cur¬ 
rent machine architectures use 2’s complement representations, this is more of a 
historic artifact. Current machine architectures use 2’s complement representations 
because these do not suffer from positive and negative zeros and thus enjoy unique 
representations of the same integer. Hence, CH 2 O restricts itself to implementations 
that use 2’s complement signed integers representations. 
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Integer representations in CH 2 O can solely differ with respect to endianness (the 
order of the bits). The function endianize takes a list of bits in little endian order and 
permutes them accordingly. We allow endianize to yield an arbitrary permutation 
and thus we not just support big- and little-endian, but also mixed-endian variants. 

Definition 4.4 Given an integer type r,, the integer encoding functions _ : t, : Z —> 
list bool and (_)T-i : list bool —^ Z are defined as follows: 

X : si k := endianize k (x as little endian 2’s complement) 

{P)si k ■= of little endian 2’s complement (deendianize k jf) 

Lemma 4.5 The integer encoding functions are inverses. That means: 

1. We have {x : = x and \x : t\\ = rank_size t, provided that x : t\. 

2. We have {13)^ ■ x\ = ft and '■ t\ provided that |/3| = rank_size t\. 


4.2 Definition of types 

We support integer, pointer, function pointer, array, struct, union and void types. 
The translation that we have described in translates more complicated types, 

such as typedefs and enums, into these simplified types. This translation also alle¬ 
viates other simplifications of our simplified definition of types, such as the use of 
unnamed struct and union fields. Floating point types and type qualifiers like const 
and volatile are not supported. 

All definitions in this section are implicitly parametrized by an integer coding 
environment with ranks K (Definition 14.21) . 

Definition 4.6 Tags t 6 tag (sometimes called struct/union names) and function 
names f e funname are represented as strings. 

Definition 4.7 Types consist of point-to types, base types and full types. These are 
inductively defined as: 

Tp, Up e ptrtype ::= r | any | f — >• t 
rb,crb G basetype ::= t, \ Tp* | void 

T,a E type ::= Tb | T[n] | struct t \ union t 

The three mutual inductive parts of types correspond to the different components 
of the memory model. Addresses and pointers have point-to types (Definitions 16.81 
and l6.10|) . base values have base types (Definition 1 6. 401) . and memory trees and values 
have full types (Definitions 16.251 and 16.4(11) . 

The void type of C is used for two entirely unrelated purposes: void is used for 
functions without return type and void* is used for pointers to objects of unspecified 
type. In CH 2 O this distinction is explicit in the syntax of types. The type void is 
used for function without return value. Like the mathematical unit type it has one 
value called nothing (Definition 16.391) . The type any* is used for pointers to objects 
of unspecified type. 

Unlike more modern programming languages C does not provide first class func¬ 
tions. Instead, C provides function pointers which are just addresses of executable 
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code in memory instead of closures. Function pointers can be used in a way similar 
to ordinary pointers: they can be used as arguments and return value of functions, 
they can be part of structs, unions and arrays, etc. 

The C language sometimes allows function types to be used as shorthands for 
function pointers, for example: 

void sortCint *p, int len, int compare(int,int)); 

The third argument of sort is a shorthand for int (*compare) (int, int) and is 
thus in fact a function pointer instead of a function. We only have function pointer 
types, and the third argument of the type of the function sort thus contains an 
additional *: 

[(signed int)*, signed int, (signed int —>■ signed int)*] —^ void. 

Struct and union types consist of just a name, and do not contain the types of 
their fields. An environment is used to assign fields to structs and unions, and to 
assign argument and return types to function names. 

Definition 4.8 Type environments are defined as: 

r 6 env := (tag ^fin list type) x (types of struct/union fields) 

(funname ^fin (list type x type)). (types of functions) 

The functions dorritag : env 'Pfin(tag) and domfunname : env —> 'Pfin(funname) yield 
the declared structs and unions, respectively the declared functions. We implicitly 
treat environments as functions tag —>fin list type and funname ^fin (list type x type) 
that correspond to underlying finite partial functions. 

Struct and union names on the one hand, and function names on the other, have 
their own name space in accordance with the Cll standard [271 6.2.3pl]. 

Notation 4.9 We often write an environment as a mixed sequence of struct and 
union declarations t : t, and function declarations / : (r, t). This is possible because 
environments are finite. 

Since we represent the fields of structs and unions as lists, fields are nameless. 
For example, the C type struct SI { int x; struct SI *p; } is translated into 
the environment SI : [signed int,struct SI*]. 

Although structs and unions are semantically very different (products versus 
sums, respectively), environments do not keep track of whether a tag has been used 
for a struct or a union type. Structs and union types with the same tag are thus 
allowed. The translator in [37113^ forbids the same name being used to declare both 
a struct and union type. 

Although our mutual inductive syntax of types already forbids many incorrect 
types such as functions returning functions (instead of function pointers), still some 
ill-formed types such as int [0] are syntactically valid. Also, we have to ensure that 
cyclic structs and unions are only allowed when the recursive definition is guarded 
through pointers. Guardedness by pointers ensures that the sizes of types are finite 
and statically known. Consider the following types: 
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struct listl { int hd; struct listl tl; /* illegal */ 

struct list2 { int hd; struct list2 *p_tl; /* legal */ 

The type declaration struct listl is illegal because it has a reference to itself. 
In the type declaration struct list2 the self reference is guarded through a pointer 
type, and therefore legal. Of course, this generalizes to mutual recursive types like: 

struct tree { int hd; struct forest *p_children; ]■; 
struct forest { struct tree *p_hd; struct forest *p_tl; }■; 


Definition 4.10 The following judgments are dehned by mutual induction: 

— The judgment T h* Tp describes point-to types Tp to which a pointer may point: 

ri-«T ri-*r rhbTb ri-r nAO 
T h* any T h* t —1 r T h* rb T h* r[n] 

r h* struct t r h* union t 

— The judgment T hb rb describes valid base types rb: 

r b* rp 

T bb r, r bb rp!i! r bb void 

— The judgment T b r describes valid types r: 

Tbbrb Tbr nAO t E domtag T t E domtag T 

r b rb r b r[n] T b struct t T b union t 

Definition 4.11 The judgment b T describes well-formed environments T. It is 
inductively dehned as: 

bT Tbr rAe ti domtag T bT Tb^f rb*r f i dornfanname T 
b 0 b t : r, r b / : (f, r), r 

Note that Tbr does not imply b T. Most results therefore have b T as a premise. 
These premises are left implicit in this paper. 

In order to support (mutually) recursive struct and union types, pointers to 
incomplete struct and union types are permitted in the judgment T b* rp that 
describes types to which pointers are allowed, but forbidden in the judgment Tbr 
of validity of types. Let us consider the following type declarations: 

struct S2 ■[ struct S2 x; ]■; /* illegal */ 

struct S3 { struct S3 *p; I; /* legal */ 

Well-formedness b T of the environment T := S3 : [struct S3=i!] can be derived 
using the judgments 0 b* struct S3, 0 bb struct S3=i!, 0 b struct S3*, and thus b T. 
The environment S2 : [struct S2] is ill-formed because we do not have 0 b struct S2. 

The typing rule for function pointers types is slightly more delicate. This is best 
illustrated by an example: 

union U { int i; union U (*f) (union U); I; 
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This example displays a recursive self reference to a union type through a function 
type, which is legal in C because function types are in fact pointer types. Due to 
this reason, the premises of T h* t —i t are T h* r and T h* t instead of T h r and 
r h T. Well-formedness of the above union type can be derived as follows: 


0 h* union U 0 h* union U 
0 h* union U —>■ union U 

0 hb signed int 0 Hb (union U ->■ union U)* 

hr 0 h signed int 0 h (union U —> union U)=i= 

h U : [signed int, (union U —> union U)=i=], T 

In order to dehne operations by recursion over the structure of well-formed types 
(see for example Dehnition 16.451 which turns a sequence of bits into a value), we 
often need to perform recursive calls on the types of helds of structs and unions. In 
Coq we have defined a custom recursor and induction principle using well-founded 
recursion. In this paper, we will use these implicitly. 

Affeldt et al. [T][2] have formalized non-cyclicity of types using a complex con¬ 
straint on paths through types. Our definition of validity of environments (Defini¬ 
tion Km follows the structure of type environments, and is more easy to use (for 
example to implement the aforementioned recursor and induction principle). 

There is a close correspondence between array and pointer types in C. Arrays are 
not first class types, and except for special cases such as initialization, manipulation 
of arrays is achieved via pointers. We consider arrays as first class types so as to 
avoid having to make exceptions for the case of arrays all the time. 

Due to this reason, more types are valid in CH 2 O than in Cll. The translator 
in resolves exceptional cases for arrays. For example, a function parameter 

of array type acts like a parameter of pointer type in Cll EZl 6.7.6.30. 

void f(int a [10]); 

The corresponding type of the function f is thus (signed int)* ^ void. Note that 
the type (signed int) [10] —^ void is also valid, but entirely different, and never gener¬ 
ated by the translator in EZIES]. 


4.3 Implementation environments 

We finish this section by extending integer coding environments to describe imple¬ 
mentation defined properties related the layout of struct and union types. The au¬ 
thor’s PhD thesis [33] also considers the implementation-defined behavior of integer 
operations (such as addition and division) and defines inhabitants of this interface 
corresponding to actual computing architectures. 

Definition 4.12 A implementation environment with ranks K consists of an integer 
coding environment with ranks K and functions: 

sizeofr : type —)• N alignofp : type —>• N fieldsizesp : list type —^ list N 

® The array size is ignored unless the static keyword is used. In case f would have the 
prototype void f(int aCstatic 10]), the pointer a should provide access to an array of at 
least 10 elements 1271 6.7.6.3]. The static keyword is not supported by CH 2 O. 
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These functions should satisfy: 


sizeofp {si k) = rank_size k 

sizeofp (rp=r) A 0 sizeofp void A 

sizeofp (T[n]) = n* sizeofp t 


sizeofp (struct t) = E fieldsizesp f 

if r t = T 

sizeofp Ti < Zi and |t| = j^l 

if fieldsizesp f = z, for each i < |t| 

sizeofp Ti < sizeofp (union t) 

if r t = T, for each i < |t| 

alignofp T 1 alignofp (T[n]) 


alignofp Ti 1 alignofp (struct t) 

if r t = T, for each i < |t| 

alignofp Ti 1 alignofp (union t) 

if r t = T, for each i < |t| 

alignofp T 1 sizeofp t 

if ri-T 

alignofp Ti 1 offsetofp T* 

if r h T, for each i < |t| 


Here, we let offsetofrr i denote Ej<i(fieldsizesr f)j. The functions sizeofp, alignofp, 
and fieldsizesr should be closed under weakening of F. 

Notation 4.13 Given an implementation environment, we let: 

bitsizeofp T := sizeofp t ■ char_bits 
bitoffsetofp T j := offsetofp t j ■ char_bits 
fieldbitsizesp t := fieldsizesp r ■ char_bits 

We let sizeofp r specify the number of bytes out of which the object representation 
of an object of type r constitutes. Objects of type r should be allocated at addresses 
that are a multiple of alignofp t. We will prove that our abstract notion of addresses 
satisfies this property (see Lemma [6.1811 . The functions sizeofp, alignofp correspond 
to the sizeof and _Alignof operators |271 6.5.3.4], and offsetofp corresponds to the 
offsetof macro mi 7.19p3]. The list fieldsizesp t specifies the layout of a struct 
type with fields f as follows: 


sizeofp tq sizeofp ti 


To 


Tl 



I (fieldsizesp t)o | (fieldsizesp r)i | 
offsetofp T 0 offsetofp f 1 offsetofp f 2 

5 Permissions and separation algebras 

Permissions control whether memory operations such as a read or store are allowed 
or not. In order to obtain the highest level of precision, we tag each individual bit in 
memory with a corresponding permission. In the operational semantics, permissions 
have two main purposes: 

— Permissions are used to formalize the sequence point restriction which assigns 
undefined behavior to programs in which an object in memory is modified more 
than once in between two sequence points. 
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— Permissions are used to distinguish objects in memory that are writable from 

those that are read-only {const qualified in C terminology). 

In the axiomatic semantics based on separation logic, permissions play an impor¬ 
tant role for share aeeounting. We use share accounting for subdivision of permissions 
among multiple subexpressions to ensure that: 

— Writable objects are unique to each subexpression. 

— Read-only objects may be shared between subexpressions. 

This distinction is originally due to Dijkstra [16] and is essential in separation 
logic with permissions m- The novelty of our work is to use separation logic with 
permissions for non-determinism in expressions in C. Share accounting gives rise to 
a natural treatment of C’s sequence point restriction. 

Separation algebras as introduced by Calcagno et al. [13] abstractly capture com¬ 
mon structure of subdivision of permissions. We present a generalization of separa¬ 
tion algebras that is well-suited for C verification in Coq and use this generalization 
to build the permission system and memory model compositionally. The permission 
system will be constructed as a telescope of separation algebras: 

perm := £(C(Q)) -b 

non-const qualified const qualified 

Here, Q is the separation algebra of fractional permissions, C is a functor that extends 
a separation algebra with a counting component, and £ is a functor that extends a 
separation algebra with a lockable component (used for the sequence point restric¬ 
tion). This section explains these functors and their purposes. 


5.1 Separation logic and share accounting 

Before we will go into the details of the CH 2 O permission system, we briefly introduce 
separation logic. Separation logic m is an extension of Hoare logic that provides 
better means to reason about imperative programs that use mutable data structures 
and pointers. The key feature of separation logic is the separating conjunction P * Q 
that allows one to subdivide the memory into two disjoint parts: a part described by 
P and another part described by Q. The separating conjunction is most prominent 
in the frame rule. 

{P}s{Q} 

{P*R}s{Q* R} 

This rule enables local reasoning. Given a Hoare triple {P} s {Q}, this rule allows 
one to derive that the triple also holds when the memory is extended with a disjoint 
part described by R. The frame rule shows its merits when reasoning about functions. 
There it allows one to consider a function in the context of the memory the function 
actually uses, instead of having to consider the function in the context of the entire 
program’s memory. However, already in derivations of small programs the use of the 
frame rule can be demonstratec0: 

^ Contrary to traditional separation logic, we do not give local variables a special status of 
being stack allocated. We do so, because in C even local variables are allowed to have pointers 
to them. 
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{x !->■ 0}x:=10 {x !->■ 10} {y i->- 0}y:=12 {y >->■ 12} 

{x I— > 0 * y I— > 0} x: =10 {x i—> 10 * y i—> 0} {x i—> 10 * y i—> 0} y :=12 {x i—> 10 * y i—> 12} 

{x I— > 0 * y H-^ 0} X: =10; y: =12 {x i—>■ 10 * y i—^ 12} 

The singleton assertion a i—i d denotes that the memory consists of exactly one 
object with value v at address a. The assignments are not considered in the context 
of the entire memory, but just in the part of the memory that is used. 

The key observation that led to our separation logic for C, see also mm, is the 
correspondence between non-determinism in expressions and a form of concurrency. 
Inspired by the rule for the parallel composition [l 6 ] , we have rules for each operator 
® that are of the following shape. 

{^’i}ei{Qi} {P2}e2{Q2} 

{Pi * P 2 } ei @ 62 {Qi * Q 2 } 

The intuitive idea of this rule is that if the memory can be subdivided into 
two parts in which the subexpressions ei and 62 can be executed safely, then the 
expression ei@e 2 can be executed safely in the whole memory. Non-interference of the 
side-effects of ei and 62 is guaranteed by the separating conjunction. It ensures that 
the parts of the memory described by Pi and P 2 do not have overlapping areas that 
will be written to. We thus effectively rule out expressions with undehned behavior 
such as (x = 3) + (x = 4) (see Section [T 6 l for discussion). 

Subdividing the memory into multiple parts is not a simple operation. In order 
to illustrate this, let us consider a shallow embedding of assertions of separation 
logic P,Q : mem —> Prop (think of mem as being the set of finite partial function 
from some set of object identifiers to some set of objects. The exact definition in 
the context of CH 2 O is given in Definition 16.2611 . In such a shallow embedding, one 
would define the separating conjunction as follows: 

P * Q := Am . 3mi m 2 . m = mi U m 2 A P mi A Q m 2 . 

The operation U is not the disjoint union of finite partial functions, but a more 
fine grained operation. There are two reasons for that. Firstly, subdivision of memo¬ 
ries should allow for partial overlap, as long as writable objects are unique to a single 
part. For example, the expression x + x has defined behavior, but the expressions 
x + (x = 4) and (x = 3) + (x = 4) have not. 

We use separation logic with permissions in to deal with partial overlap of 
memories. That means, we equip the singleton assertion a v with a permission 7 . 
The essential property of the singleton assertion is that given a writable permission 
Jjjj there is a readable permission 7 ^ with: 

(a v) ^ {a v) =t= (a v). 

The above property is an instance of a slightly more general property. We consider 
a binary operation U on permissions so we can write: 

(a v) (a v) * (a ti). 
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Secondly, it should be possible to subdivide array, struct and union objects into 
subobjects corresponding to their elements. For example, in the case of an array 
int a [2], the expression (a[0] = 1) + (a[l] = 4) has dehned behavior, and we 
should be able to prove so. The essential property of the singleton assertion for an 
array [ 7 / 0 ,..., value is: 

(a array [tro,..., ]) (a[ 0 ] 7 ;o) * ' •' * (o[?^ - 1 ] v„_i). 

This paper does not describe the CH 2 O separation logic and its shallow embed¬ 
ding of assertions. These are described in the author’s PhD thesis [33]. Instead, we 
consider just the operations U on permissions and memories. 


5.2 Separation algebras 

As shown in the previous section, the key operation needed to define a shallow em¬ 
bedding of separation logic with permissions is a binary operation U on memories 
and permissions. Calcagno et al. introduced the notion of a separation algebra [T3] so 
as to capture common properties of the U operation. A separation algebra (A, 0, U) 
is a partial cancellative commutative monoid (see Definition 15.II for our actual defi¬ 
nition). Some prototypical instances of separation algebras are: 

— Finite partial functions (A —>'fin S,0, U), where 0 is the empty finite partial 
function, and U the disjoint union on finite partial functions. 

— The Booleans (bool, false, V). 

— Boyland’s fractional permissions ([0, 1 ]q, 0,+) where 0 denotes no access, 1 de¬ 
notes writable access, and 0 < _ < 1 denotes read-only access mm- 

Separation algebras are also closed under various constructs (such as products 
and finite functions), and complex instances can thus be built compositionally. 

When formalizing separation algebras in the Coq proof assistant, we quickly ran 
into some problems: 

— Dealing with partial operations such as U is cumbersome, see Section 18.31 

— Dealing with subset types (modeled as E-types) is inconvenient. 

— Operations such as the difference operation \ cannot be defined constructively 
from the laws of a separation algebra. 

In order to deal with the issue of partiality, we turn U into a total operation. 
Only in case x and y are disjoint, notation x J- y, we require x U y to satisfy the 
laws of a separation algebra. Instead of using subsets, we equip separation algebras 
with a predicate valid : A —> Prop that explicitly describes a subset of the carrier A. 
Lastly, we explicitly add a difference operation \. 

Definition 5.1 A separation algebra consists of a type A, with: 

— An element 0 : A 

— A predicate valid : A —^ Prop 

— Binary relations _L, C : A —)• A —^ Prop 

— Binary operations U, \ : A ^ A ^ A 

Satisfying the following laws: 
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1. If valid X, then 0 _L a; and 0 U a; = a; 

2. If a; -L y, then y .L x and x U y = y U x 

3. If a: _L y and x U y J- z, then y ± z, x A- y U z, and x U {y U z) = {x D y) U z 

4. li z A- X, z A- y and z iA x = z VA y, then x = y 

5. If a; -L i/, then valid x and valid {x U y) 

6 . If a; _L y and x U y = 9, then a; = 0 

7. If a: _L y, then x C x U y 

8 . If a: C y, then a: _L y \ a; and x U y \ x = y 

Laws [T}3] describe the traditional laws of a separation algebra: identity, commu¬ 
tativity, associativity and cancellativity. Law^ensures that valid is closed under the 
U operation. Law [B] describes positivity. Laws [3 and [S] fully axiomatize the C relation 
and \ operation. Using the positivity and cancellation law, we obtain that C is a 
partial order in which U is order preserving and respecting. 

In case of permissions, the 0 element is used to split objects of compound types 
(arrays and structs) into multiple parts. We thus use separation algebras instead of 
permission algebras m, which are a variant of separation algebras without an 0 
element. 

Definition 5.2 The Boolean separation algebra bool is dehned as: 

valid x := True 0 := false 

X A- y := -'X \/ ^y x U y ■.= x V y 

X C y := X ^ y x\y ■.= x /\-iy 

In the case of fractional permissions [0, 1 ]q the problem of partiality and subset 
types already clearly appears. The U operation (here -|-) can ‘overflow’. We remedy 
this problem by having all operations operate on pre-terms (here Q) and the predicate 
valid describes validity of pre-terms (here 0 < _ < 1). 

Definition 5.3 The fractional separation algebra Q is defined as: 

valid a;:=0<a;<l 0:=O 

x±y:=0<x,y A x + y<l x U y ■.= x + y 

xQy.= 0<x<y<l x \ y := x — y 

The version of separation algebras by Klein et al. [22 in Isabelle also models U 
as a total operation and uses a relation T. There are some differences: 

— We include a predicate valid to prevent having to deal with subset types. 

— They have weaker premises for associativity (lawlSj, namely x A- y, y A- z and 
X A- z instead of x A^ y and x U y A- z. Ours are more natural, e.g. for fractional 
permissions one has 0.5 T 0.5 but not 0.5-1-0.5 T 0.5, and it thus makes no sense 
to require 0.5 U (0.5 U 0.5) = (0.5 U 0.5) U 0.5 to hold. 

— Since Coq (without axioms) does not have a choice operator, the \ operation 
cannot be defined in terms of U. Isabelle has a choice operator. 

Dockins et al. m have formalized a hierarchy of different separation algebras in 
Coq. They have dealt with the issue of partiality by treating U as a relation instead 
of a function. This is unnatural, because equational reasoning becomes impossible 
and one has to name all auxiliary results. 

Bengtson et al. [ 6 ] have formalized separation algebras in Coq to reason about 
object-oriented programs. They have treated U as a partial function, and have not 
defined any complex permission systems. 
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5.3 Permissions 

We classify permissions using permission kinds. 

Definition 5.4 The lattice of permission kinds (pkind, C) is defined as: 

Writable 

/ \ 

Readable Locked 
\ / 

Existing 

_L 

The order ki C k 2 expresses that ki allows fewer operations than ^ 2 . This orga¬ 
nization of permissions is inspired by Leroy et al. . The intuitive meaning of the 
permission kinds is as follows: 

— Writable. Writable permissions allow reading and writing. 

— Readable. Read-only permissions allow solely reading. 

— Existing. Existence permissions El are used for objects that are known to exist 
but whose value cannot be used. Existence permissions are essential because C 
only permits pointer arithmetic on pointers that refer to objects that have not 
been deallocated (see Section [33] for discussion). 

— Locked. Locked permissions are used to formalize the sequence point restriction. 
When an object is modified during the execution of an expression, it is tem¬ 
porarily given a locked permission to forbid any read/write accesses until the 
next sequence point. 

For example, in (x = 3) + *p; the assignment x = 3 locks the permissions of 
the object x. Since future read/write accesses to x are forbidden, accessing *p 
results in undefined in case p points to x. At the sequence point ;, the original 
permission of x is restored. 

Locked permissions are different from existence permissions because the oper¬ 
ational semantics can change writable permissions into locked permissions and 
vice versa, but cannot do that with existing permissions. 

— T. Empty permissions allow no operations. 

In our separation logic we do not only have control which operations are allowed, 
but also have to deal with share accounting. 

— We need to subdivide objects with writable or read-only permission into multiple 
parts with read-only permission. For example, in the expression x + x, both 
subexpressions require x to have at least read-only permission. 

— We need to subdivide objects with writable permission into a part with existence 
permission and a part with writable permission. For example, in the expression 
*(p + 1) = (*p = 1), the subexpression *p = 1 requires *p to have writable 
permission, and the subexpression * (p + 1 ) requires *p to have at least existence 
permission so as to perform pointer arithmetic on p. 

When reassembling subdivided permissions (using U), we need to know when the 
original permission is reobtained. Therefore, the underlying permission system needs 
to have more structure, and cannot consist of just the permission kinds. 
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♦ ( 0 , 1 ) 


LockOl 



1 

Readable 

0 


Figure 1 The CH 2 O permission system. 


Definition 5.5 CH 2 O permissions perm are defined as: 


Lockable SA Counting SA Fractional SA 

-^ i / --' 

7 6 perm := r(C(Q)) + 




non-const qualified 


const qualified 


where C{A) := {♦, 0 } x A and C(A) := Q x A. 

The author’s PhD thesis gives the exact definition of the separation algebra 
operations on permissions by defining these one by one for the counting separation 
algebra C, the lockable separation algebra £, and the separation algebra on sums +. 
This section gives a summary of the important aspects of the permission system. 

We combine fractional permissions to account for read-only/writable permissions 
with counting permissions to account for the number of existence permissions that 
have been handed out. Counting permissions have originally been introduced by 
Bornat et al. HU. The annotations {♦, 0} describe whether a permission is locked ♦ 
or not 0. Only writable permission have a locked variant. 

Const permissions are used for objects declared with the const qualifier. Modify¬ 
ing an object with const permissions results in undefined behavior. Const permissions 
do not have a locked variant or a counting component as they do not allow writing. 

Figure [U indicates the valid predicate by the areas marked green and displays 
how the elements of the permission system project onto their kinds. The operation 
U is defined roughly as the point-wise addition and \ as point-wise subtraction. 

We will define an operation | : perm —> perm to subdivide a writable or read-only 
permission into read-only permissions. 

{ 0(0.5 ■ x, 0.5 ■ y) if 7 = 0(a;, y) 

0.5 • a: if 7 = a: e Q 

7 otherwise, dummy value 

Given a writable or read-only permission 7 , the subdivided read-only permission 
enjoys ^7 _L \'y and ^7 U §7 = 7 . 

The existence permission token := 0(—1, 0) is used in combination with the 
\ operation to subdivide a writable permission 7 into a writable permission 7 \ 
token and an existence permission token. We have 7 U (7 \ token) = 7 by law [5] 
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of separation algebras. Importantly, only objects with 0(0, 1) permission can be 
deallocated, whereas objects with 7 \ token permission cannot (see Definition IB.581) 
because expressions such as (p == p) + (free(p), 0 ) have undehned behavior. 


5.4 Extended separation algebras 


We extend separation algebras with a split operation | and predicates to distinguish 
permissions in our memory model. 

Definition 5.6 An extended separation algebra extends a separation algebra with: 

— Predicates splittable, unmapped, exclusive : A —^ Prop 

— A unary operation | : A —^ A 

Satisfying the following laws: 

9. If a: A x, then splittable (x U x) 

10. If splittable x, then ^x A ^x and ^x U ^x = x 

11. If splittable y and x C y, then splittable x 

12. li X A- y and splittable (a; U y), then |(a; VA y) = ^xiJ \y 

13. unmapped 0, and if unmapped a:, then valid x 

14. If unmapped y and a: C y, then unmapped x 

15. If a; A y, unmapped x and unmapped y, then unmapped (a: U y) 

16. exclusive x iff valid x and for all y with a; A y we have unmapped y 

17. Not both exclusive x and unmapped x 

18. There exists an x with valid x and not unmapped x 

The 4-operation is partial, but described by a total function whose result ^x is 
only meaningful if splittable x holds. Law ensures that splittable permissions are 
inhnitely splittable, and law [12] ensures that | distributes over U. 

The predicates unmapped and exclusive associate an intended semantics to ele¬ 
ments of a separation algebra. Let us consider fractional permissions to indicate the 
intended meaning of these predicates. 


Definition 5.7 The fractional separation algebra Q is extended with: 

splittable a: := 0 < a; < 1 -x := 0.5 ■ x 

unmapped x := x = Q exclusive a; := a: = 1 


Remember that permissions will be used to annotate each individual bit in mem¬ 
ory. Unmapped permissions are on the bottom: they do not allow their bit to be used 
in any way. Exclusive permissions are on the top: they are the sole owner of a bit 
and can do anything to that bit without affecting disjoint bits. 

Fractional permissions have exactly one unmapped element and exactly one ex¬ 
clusive element, but in the CH 2 O permission system this is not the case. The elements 
of the CH 2 O permission system are classihed as follows: 


unmapped 

exclusive 

Examples 

y 

y 

Writable and Locked permissions 

Readable permissions 

The 0 permission and Existing permissions 
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In order to abstractly describe bits annotated with permissions we define the 
tagged separation algebra In its concrete use 7^^j(perm) in the memory model 

fDefinition l6.21L the elements ( 7 , b) consist of a permission 7 € perm and bit b 6 bit. 
We use the symbolic bit / that represents indeterminate storage to ensure that bits 
with unmapped permissions indeed have no usable value. 


Definition 5.8 Given a separation algebra A and a set of tags T with default tag 
t E T, the tagged separation algebra := A x T over A is defined as: 


valid (x, y) 
0 

{x, y) ± {x\ y') 

{x, y) U {x', y') 

splittable {x, y) 

lix, y) 
unmapped {x, y) 
exclusive (a;, y) 


valid X A (unmapped x ^ y = t) 

( 0 , t) 

X J- x' A (unmapped xV y = y' V unmapped x') 

A (unmapped x ^ y = t) A (unmapped x' ^ y' = t) 

f (a; U x', y') if y = t 
\{xiJx',y) otherwise 

splittable X A (unmapped x ^ y = t) 

y) 

unmapped x Ay = t 
exclusive x 


The definitions of the omitted relations and operations are as expected. 


6 The memory model 

This section defines the CH 2 O memory model whose external interface consists of 
operations with the following types: 

lookupp : addr mem —> option val 

forcer : addr mem —> mem 

insertp : addr mem —^ val —^ mem 

writablep : addr —> mem —> Prop 
lockp : addr —> mem —> mem 
unlock : lockset —> mem —> mem 
allocp : index —> val —> bool —> mem —> mem 
dom : mem —> Pfin(index) 
freeable : addr —> mem —^ Prop 
free : index —>• mem —^ mem 

Notation 6.1 We let m{a)r := lookupp a m and m{a := t;)r := insertp a v m. 

Many of these operations depend on the typing environment F which assigns 
fields to structs and unions (Definition 14.81) . This dependency is required because 
these operations need to be aware of the layout of structs and unions. 
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The operation m{a)r yields the value stored at address a in memory m. It fails 
with ± if the permissions are insufficient, effective types are violated, or a is an end-of- 
array address. Reading from (the abstract) memory is not a pure operation. Although 
it does not affect the memory contents, it may affect the effective types mi 6.5p6-7]. 
This happens for example in case type-punning is performed (see Section RUHl . This 
impurity is factored out by the operation forcer « rn. 

The operation m{a := v)r stores the value v at address a in memory m. A store 
is only permitted in case permissions are sufficient, effective types are not violated, 
and a is not an end-of-array address. The proposition writablera rn describes the 
side-conditions necessary to perform a store. 

After a successful store, the operation lockp a m is used to lock the object at 
address a in memory m. The lock operation temporarily reduces the permissions 
to Locked so as to prohibit future accesses to a. Locking yields a formal treatment 
of the sequence point restriction (which states that modifying an object more than 
once between two sequence points results in undehned behavior, see Section ESI)- 

The operational semantics accumulates a set 6 lockset of addresses that have 
been written to (Definition 16.5411 and uses the operation unlock Q m at the subse¬ 
quent sequence point (which may be at the semicolon that terminates a full expres¬ 
sion). The operation unlock Q m restores the permissions of the addresses in Q and 
thereby makes future accesses to the addresses in possible again. The author’s 
PhD thesis ES] describes in detail how sequence points and locks are treated in the 
operational semantics. 

The operation allocp o v fj, m allocates a new object with value v in memory m. 
The object has object identiher o dom m which is non-deterministically chosen 
by the operation semantics. The Boolean p expresses whether the new object is 
allocated by malloc. 

Accompanying allocr, the operation free o ra deallocates a previously allocated 
object with object identifier o in memory ra. In order to deallocate dynamically 
obtained memory via free, the side-condition freeable a m describes that the per¬ 
missions are sufficient for deallocation, and that a points to a malloced object. 


6.1 Representation of pointers 

Adapted from CompCert mM, we represent memory states as hnite partial func¬ 
tions from object identifiers to objects. Each local, global and static variable, as well 
as each invocation of malloc, is associated with a unique object identifier of a sep¬ 
arate object in memory. This approach separates unrelated objects by construction, 
and is therefore well-suited for reasoning about memory transformations. 

We improve on CompCert by modeling objects as structured trees instead of 
arrays of bytes to keep track of padding bytes and the variants of unions. This is 
needed to faithfully describe Cll’s notion of effective types (see page E] of Section [T] 
for an informal description). This approach allows us to describe various undefined 
behaviors of Cll that have not been considered by others (see Sections 13.II and Ifi.lID . 

In the CompCert memory model, pointers are represented as pairs (o, i) where o 
is an object identifier and * is a byte offset into the object with object identifier o. 
Since we represent objects as trees instead of as arrays of bytes, we represent pointers 
as paths through these trees rather than as byte offsets. 
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Definition 6.2 Object identifiers o E index are elements of a fixed countable set. In 
the Coq development we use binary natural numbers, but since we do not rely on 
any properties apart from countability, we keep the representation opaque. 

We first introduce a typing environment to relate the shape of paths representing 
pointers to the types of objects in memory. 

Definition 6.3 Memory typing environments A E memenv are hnite partial func¬ 
tions index —^fin (type x bool). Given a memory environment A: 

1. An object identifier o has type t, notation A h o : r, if A o = (t, /3) for a /3. 

2. An object identifier o is alive, notation A h o alive, if Ao = (t, false) for a r. 

Memory typing environments evolve during program execution. The code below 
is annotated with the corresponding memory environments in red. 

short X; 

Ai = {oi i-i- (signed short, false)} 
int *p; 

^2 = {oi (signed short, false), 02 (signed int*, false)} 
p = malloc(sizeof(int)); 

(^3 = {oi I—> (signed short, false), 02 (signed int*, false), 03 1 -^ (signed int, false)} 
free(p); 

A 4 = {oi 1 —^ (signed short, false), 02 i-t (signed int*, false), 03 i-t (signed int, true)} 

Here, oi is the object identifier of the variable x, 02 is the object identifier of the 
variable p and 03 is the object identifier of the storage obtained via malloc. 

Memory typing environments also keep track of objects that have been deallo¬ 
cated. Although one cannot directly create a pointer to a deallocated object, existing 
pointers to such objects remain in memory after deallocation (see the pointer p in 
the above example). These pointers, also called dangling pointers, cannot actually 
be used. 

Definition 6.4 References, addresses and pointers are inductively defined as: 

tIti] I struct t , union t . . 

r E refseg ::= '—> i \ '- > i \ '- >q i with q E {o, •} 

r E ref := list refseg 
a E addr ::= (o : T,f,i)a>,a„ 
pE ptr NULL Up | a | 

References are paths from the top of an object in memory to some subtree of 
that object. The shape of references matches the structure of types: 

T[n] 

— The reference '— > i is used to select the fth element of a r-array of length n. 

— The reference > i is used to select the iih field of a struct t. 

— The reference '- >q i is used to select the ith variant of a union t. 

References can describe most pointers in C but cannot account for end-of-array 
pointers and pointers to individual bytes. We have therefore defined the richer notion 
of addresses. An address (o : T,f,i)a>,ap consists of: 
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— An object identifier o with type r. 

— A reference r to a subobject of type a in the entire object of type t. 

— An offset i to a particular byte in the subobject of type a (note that one cannot 
address individual bits in C). 

— The type CTp to which the address is cast. We use a points-to type in order to 
account for casts to the anonymous void* pointer, which is represented as the 
points-to type any. This information is needed to define, for example, pointer 
arithmetic, which is sensitive to the type of the address. 

In turn, pointers extend addresses with a NULL pointer NULL cTp for each type 
(Tp, and function pointers which contain the name and type of a function. 

Let us consider the following global variable declaration: 

struct S { 

union U { signed char x[2]; int y; } u; 
void *p; 

> s; 

The formal representation of the pointer (void*) (s.u.x + 2) is: 


struct S union U signed char[2] 

(Os . struct S, ^ 0 ^ 0 ^ y 0, 2)sjgned char>*any 


Here, Os is the object identifier associated with the variable s of type struct S. 

struct S union U signed char[2] 

The reference '-^ 0 '-S’, 0 '-^ 0 and byte-offset 2 describe that the 

pointer refers to the third byte of the array s.u.x. The pointer refers to an object 
of type signed char. The annotation any describes that the pointer has been cast to 
type void*. 

r 1 r union s 

The annotations q € {o, •} on references '- >q i describe whether type-punning 

is allowed or not. The annotation • means that type-punning is allowed, i.e. accessing 
another variant than the current one has defined behavior. The annotation o means 
that type-punning is forbidden. A pointer whose annotations are all of the shape o, 
and thereby does not allow type-punning at all, is called frozen. 

Definition 6.5 The freeze function | _|o : refseg ^ refseg is defined as: 


. ^[ 71 ] . . Tin] struct t . . struct t . , union t . . union t 

\ ‘ S’ 1 |o i= ' ^ 1 I ' S’ * |o i= ' ^ * I ' S-q I |o := ' * 

A reference segment r is frozen, notation frozen r, if | r |o = r. Both | _ |o and frozen 
are lifted to references, addresses, and pointers in the expected way. 

Pointers stored in memory are always in frozen shape. Definitions 16.321 and 16.411 
describe the formal treatment of effective types and frozen pointers, but for now we 
reconsider the example from Section [331 

union U { int x; short y; ]-u = { .x = 3]-; 
short *p = &u.y; 

printf ("°/,d\n" , *p) ; // Undefined 

printf ("°/,d\n" , u.y); // OK 
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Assuming the object u has object identiher Ou, the pointers &u.x, &u.y and p 
have the following formal representations: 


&u. x: 

(ou : 

: union U, 

union U 

0 ; 0 )signed int>*signed int 

'- 


(Ou : 


union U 

I 5 0 )signed short>*signed short 

feu.y: 

: union U, 

' - 


(Ou : 

: union U, 

union U 

0 ) 0 )signed short>*signed short 

p: 

^ o 


These pointers are likely to have the same object representation on actual com¬ 
puting architectures. However, due to effective types, &u.y may be used for type- 
punning but p may not. It is thus important that we distinguish these pointers in 
the formal memory model. 

The additional structure of pointers is also needed to determine whether pointer 
subtraction has dehned behavior. The behavior is only defined if the given pointers 
both point to an element of the same array object [m 6.5.6p9]. Consider: 

struct S { int a [3] ; int b [3] ; d s; 

s.a - s.b; // Undefined, different array objects 

(s.a + 3) - s.b; // Undefined, different array objects 
(s.a + 3) - s.a; // OK, same array objects 

Here, the pointers s.a + 3 and s.b have different representations in the CH 2 O 
memory model. The author’s PhD thesis [33] gives the formal definition of pointer 
subtraction. 

We will now define typing judgments for references, addresses and pointers. The 
judgment for references T h r : r v-> cr states that cr is a .subobject type of t which 
can be obtained via the reference f (see also Definition 17.HI . For example, int [2] is 

struct S 

a subobject type of struct S { int x [2] ; int y [3] ; } via '-)■ 0. 

Definition 6.6 The judgment T h r : t >-> cr describes that f is a valid reference 
from T to a. It is inductively defined as: 

V \- r : T cr[n\ i < n 
\-s:t y^T r p i-ry^a 

r h r : T >-o struct t T t = a i < \<j\ ri-r:r>-o union t T t = a i < \a\ 

Struct t union t 

I h r "->• I : T Gi r h r - >q i : r Gi 

The typing judgment for addresses is more involved than the judgment for ref¬ 
erences. Let us first consider the following example: 

int a [4] ; 

Assuming the object a has object identifier o^, the end-of-array pointer a+4 could 
be represented in at least the following ways (assuming sizeof (signed int) = 4): 

signed int[4] 

(Oa . signsd int[4], y 0, 16)signed int>*signed int 


signed int[4] 

(^a ■ signsd int[4], >■ 3,4)signecj int>*signed int 

















32 


Robbert Krebbers 


In order to ensure canonicity of pointer representations, we let the typing judg¬ 
ment for addresses ensure that the reference r of (o : r, r, i)a>,ap always refers to the 
first element of an array subobject. This renders the second representation illegal. 


Definition 6.7 The relation r >* Up, type r is pointer castable to Up, is inductively 
defined by t t, t >* unsigned char, and r >* any. 


Definition 6.8 The judgment T, A h* a : Up describes that the address a refers to 
type Tp. It is inductively defined as: 

Ahoir Thr T \- f: t a 
offset r = 0 i < sizeofp cr ■ size f sizeofp Up | i a >* Up 
r, A h (o : T,f,i)„y^^^ : Up 

Here, the helper functions offset, size : ref —>■ N are defined as: 


offset f := 


r n 

if r = f 2 '—> i 
otherwise 


size r : = 



r n 

if r = r2 '—> i 
otherwise 


We use an intrinsic encoding of syntax, which means that terms contain redun¬ 
dant type annotations so we can read off types. Functions to read off types are named 
typeof and will not be defined explicitly. Type annotations make it more convenient 
to define operations that depend on types (such as offset and size in Definition 16.81) . 
As usual, typing judgments ensure that type annotations are consistent. 

The premises i < sizeofp a ■ size r and sizeofp Up | i of the typing rule ensure that 
the byte offset i is aligned and within range. The inequality i < sizeofp cr ■ size r is 
non-strict so as to allow end-of-array pointers. 


Definition 6.9 An address (o : T,f,i)cry,a is called strict, notation T h a strict, in 
case it satisfies i < sizeofp cr ■ size f. 


The judgment r >* Op does not describe the typing restriction of cast expressions. 
Instead, it defines the invariant that each address (o : r, r, i)a>,a should satisfy. Since 
C is not type safe, pointer casting has t >* Op as a run-time side-condition: 


int x, *p = &x; 
void *q = (void*)p; 
int *ql = (int*)q; 
short *q2 = (short*)p; 
short *q3 = (short*)q; 


// OK, signed int >* any 
// OK, signed int >* signed int 
// Statically ill-typed 

// Undefined behavior, signed int signed short 


Definition 6.10 The judgment F, A h* p : (Tp describes that the pointer p refers to 
type Tp. It is inductively defined as: 

TK (Tp r,AFa:gp r/ = (f, r) 

F, A h* NULL gp : gp r,Al-«a:gp F, A h* : f-J -t 

Addresses (o : r, r "—> jA)crxcrp that point to an element of r[n] always have 
their reference point to the first element of the array, i. e. j = 0. For some operations 
we use the normalized reference which refers to the actual array element. 
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Definition 6.11 The functions index : addr —>■ index, refp : addr —^ ref, and bytep : 
addr —> N obtain the index, normalized referenee, and normalized byte offset. 


index (o : T,f,i)a>,a„ '■= o 
refp (o : t, f, i)a>,ap ■= setoffset (i sizeofp a) r 
bytep (o : t, r, i)a>^a^ '■= * mod (sizeofp a) 

Here, the function setoffset : N —t ref —> ref is defined as: 


setoffset j r := 



T n 

if r = r 2 '—> i 
otherwise 


Let us display the above definition graphically. Given an address (o : r, r, i)a>,a , 
the normalized reference and normalized byte offset are as follows: 


refp a 



bytep a 





1 -^-> 


r 


For end-of-array addresses the normalized reference is ill-typed because references 
cannot be end-of-array. For strict addresses the normalized reference is well-typed. 

Definition 6.12 The judgment A h p alive describes that the pointer p is alive, ft 
is inductively defined as: 


A h (index a) alive 

A h NULL (Tp alive Aha alive A h alive 

The judgment A h o alive on object identifiers is defined in Definition 16.31 

For many operations we have to distinguish addresses that refer to an entire 
object and addresses that refer to an individual byte of an object. We call addresses 
of the later kind byte addresses. For example: 

int X, *p = &x; // p is not a byte address 

unsigned char *q = (unsigned char*)&x; // q is a byte address 


Definition 6.13 An address (o : T,f,i)a>,a-p is a byte address if cr A (Jp. 

To express that memory operations commute (see for example Lemma [6.361) . we 
need to express that addresses are disjoint, meaning they do not overlap. Addresses 
do not overlap if they belong to different objects or take a different branch at an 
array or struct. Let us consider an example: 

union { struct { int x, y; } s; int z; ]■ ul, u2; 




















34 


Robbert Krebbers 


The pointers &ul and &u 2 are disjoint because they point to separate memory 
objects. Writing to one does not affect the value of the other and vice versa. Likewise, 
&ul. s. X and &ul. s . y are disjoint because they point to different fields of the same 
struct, and as such do not affect each other. The pointers &ul.s.x and feul.z are 
disjoint because they point to overlapping objects and thus do affect each other. 

Definition 6.14 Disjointness of references r\ and r 2 , notation fi _L r 2 , is induc¬ 
tively defined as: 

|D|o = |r2 |o i*j I n |o = I r2 |o ii= j 

cr[n] -» struct t . struct t . 

fi ^—)■ if^ ±f 2 ^^ j f 4 ri =-)■ ^ ra _L r 2 ^^ j 

Note that we do not require a special case for |ri |o 4^ |o. Such a case is 

implicit because disjointness is defined in terms of prefixes. 

Definition 6.15 Disjointness of addresses ai and 02, notation Oi _Lr 0-2, is induc¬ 
tively defined as: 

index fli 4 index 02 index Oi = index 02 refp fli -L refr 02 
oi J-r CL2 fli J-r 0'2 

both Oi and 02 are byte addresses 
index Oi = index 02 | refp ai |o = | refr 02 |o bytep ai 4 bytCp 02 

Oi J-r 02 

The first inference rule accounts for addresses whose object identifiers are dif¬ 
ferent, the second rule accounts for addresses whose references are disjoint, and the 
third rule accounts for addresses that point to different bytes of the same subobject. 

Definition 6.16 The reference bit-offset bitoffsetp : refseg —> N is defined as: 

r[n] 

bitoffsetp ('—> i) := i ■ bitsizeofp t 
bitofrsetr ("- >q i) '.= 0 

, struct 't ^ . ff. f. 

bitofrsetr (‘- > *) := bitofFsetofp t i where T t = t 

Moreover, we let bitoffsetp a := (bitoffsetp (refp a)i) -b bytep a ■ char_bits. 

Disjointness implies non-overlapping bit-offsets, but the reverse implication does 
not always hold because references to different variants of unions are not disjoint. For 
example, given the declaration union { struct { int x, y; ]- s; int z; 1 u, 
the pointers corresponding to feu.s.y and &u.z are not disjoint. 

Lemma 6.17 //r,A h ai : ai, T, A h 02 : 0 - 2 , T h { 01 , 02 } strict, oi -Lp 02 , and 
index oi 4 index 02 , then either: 

1. bitoffsetp oi -b bitsizeofp iTi < bitoffsetp 02 , or 

2. bitoffsetp 02 -b bitsizeofp 0-2 < bitoffsetp Oi. 

Lemma 6.18 (Well-typed addresses are properly aligned) //r,A h o : o, 
then (alignofp <t • char_bits) | bitoffsetp o. 
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6.2 Representation of bits 

As shown in Section [TU each object in C can be interpreted as an unsigned char 
array called the object representation. On actual computing architectures, the object 
representation consists of a sequence of concrete bits (zeros and ones). However, so 
as to accurately describe all undefined behaviors, we need a special treatment for the 
object representations of pointers and indeterminate memory in the formal memory 
model. To that end, CH 2 O represents the bits belonging to the object representations 
of pointers and indeterminate memory symbolically. 

Definition 6.19 Bits are inductively defined as: 

& e bit ::= / I 0 I 1 I (ptrp)i. 

Definition 6.20 The judgment F, A h 6 describes that a bit b is valid. It is induc¬ 
tively defined as: 

P e {0, 1 } r, A h* p : (Tp frozen p i < bitsizeofy (ffp*) 
r,Ah/ r,Ah/3 r,Ah (ptrp), 

A bit is either a concrete bit 0 or 1, the ith fragment bit (ptr p)i of a pointer p, 
or the indeterminate bit /. Integers are represented using concrete sequences of bits, 
and pointers as sequences of fragment bits. Assuming bitsizeof (signed int=r) = 32, 
a pointer p : signed int will be represented as the bit sequence (ptrp)o ... (ptrpjsi, 
and assuming bitsizeof (signed int) = 32 on a little-endian architecture, the integer 
33 : signed int will be represented as the bit sequence 1000010000000000. 

The approach using a combination of symbolic and concrete bits is similar to 
Leroy et al. [JD] and has the following advantages: 

— Symbolic bit representations for pointers avoid the need to clutter the memory 
model with subtle, implementation defined, and run-time dependent operations 
to decode and encode pointers as concrete bit sequences. 

— We can precisely keep track of memory areas that are uninitialized. Since these 
memory areas consist of arbitrary concrete bit sequences on actual machines, 
most operations on them have undefined behavior. 

— While reasoning about program transformations one has to relate the memory 
states during the execution of the source program to those during the execution 
of the target program. Program transformations can, among other things, make 
more memory defined (that is, transform some indeterminate / bits into deter¬ 
minate bits) and relabel the memory. Symbolic bit representations make it easy 
to deal with such transformations (see Section [m. 

— It vastly decreases the amount of non-determinism, making it possible to evaluate 
the memory model as part of an executable semantics [SIES]. 

— The use of concrete bit representations for integers still gives a semantics to many 
low-level operations on integer representations. 

A small difference with Leroy et al. m is that the granularity of our memory 
model is on the level of bits rather than bytes. Currently we do not make explicit use 
of this granularity, but it allows us to support bit-fields more faithfully with respect 
to the Cll standard in future work. 

Objects in our memory model are annotated with permissions. We use permission 
annotations on the level of individual bits, rather than on the level of bytes or entire 
objects, to obtain the most precise way of permission handling. 
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Definition 6.21 Permission annotated bits are defined as: 

b e pbit := 7Jj^^(perm) = perm x bit. 

In the above definition, T is the tagged separation algebra that has been defined 
in Definition 15.81 We have spelled out its dehnition for brevity’s sake. 

Definition 6.22 The judgment T, A h b describes that a permission annotated bit 
b is valid. It is inductively defined as: 

r, A h 6 valid 7 b = / in case unmapped 7 
r,Ah(7, b) 


6.3 Representation of the memory 


Memory trees are abstract trees whose structure corresponds to the shape of data 
types in C. They are used to describe individual objects (base values, arrays, structs, 
and unions) in memory. The memory is a forest of memory trees. 

Definition 6.23 Memory trees are inductively defined as: 

w e mtree ::= baser,, b | array.,, w \ structt wb | uniont (z, w, b) | union^ b. 


The structure of memory trees is close to the structure of types fPefinition 14.711 
and thus reflects the expected semantics of types: arrays are lists, structs are tuples, 
and unions are sums. Let us consider the following example: 


struct S { 



union U { signed 

char X [2] ; 

int y; } u; void *p; 

}s={ .u={ .x= 

^ {33,34} }, 

.p = s.u.x + 2 }; 


The memory tree representing the object s with object identifier Os may be as 
follows (permissions are omitted for brevity’s sake, and integer encoding and padding 
are subject to implementation defined behavior): 


structs 


signed char; I 



, Struct S union U signed char[2] 

P — (Os • struct S, ' y 0 0 y 0, 2)sjgped char>^,any 


The representation of unions requires some explanation. We considered two kinds 
of memory trees for unions: 

— The memory tree uniorit (i, w, b) represents a union whose variant is i. Unions of 
variant i can only be accessed through a pointer to variant i. This is essential for 
effective types. The list b represents the padding after the element w. 
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— The memory tree unions b represents a union whose variant is yet unspecihed. 
Whenever the union is accessed through a pointer to variant i, the list b will be 
interpreted as a memory tree of the type belonging to the ith variant. 

The reason that we consider unions unions b with unspecihc variant at all is that 
in some cases the variant cannot be known. Unions that have not been initialized do 
not have a variant yet. Also, when a union object is constructed byte-wise through 
its object representation, the variant cannot be known. 

Although unions are tagged in the formal memory, actual compilers implement 
untagged unions. Information about variants should thus be internal to the formal 
memory model. In Section [72] we prove that this is indeed the case. 

The additional structure of memory trees, namely type annotations, variants 
of unions, and structured information about padding, can be erased by flattening. 
Flattening just appends the bytes on the leaves of the tree. 

Definition 6.24 The flatten operation (_) : mtree —>■ list pbit is defined as: 

base^-t b := b array^ w := wg ... W|^|_i 

strucUwb := (uJjJbo)... (uJjjsPTb|i5|_i) uniorit (j, w, b) := uJb uniontb:=b 

The flattened version of the memory tree representing the object s in the previous 
example is as follows: 

10000100 01000100 //////// mill:}} (ptrp)o (ptr p)i... (ptr p)3i 

Definition 6.25 The judgment F, A h in : t describes that the memory tree w has 
type T. It is inductively dehned as: 

r kb Tb F, A h b |b| = bitsizeofp Tb F, A h w : t \w\ = n Q 
F, A h base.ji, b : rt F, A h array^ w : r[n] 

Vt = f F, AI-1 (;:t 

Vi. (F, A h bi hi all / |bi| = (fieldbitsizesp T)i — bitsizeofr Ti) 

F, A h structj wh : struct t 

Vt = T i<|f| r,Ahin:T, F,Ahb ball/ 
bitsizeofr (union t) = bitsizeofp tv + [b] -lunmapped {wh) 

F, A h uniont (i, w, b) : union t 

V t = T F, A h b |b| = bitsizeofr (union t) 
r, A h uniont b : union t 

Although padding bits should be kept indeterminate (see Section IHTTIl . padding 
bits are explicitly stored in memory trees for uniformity’s sake. The typing judgment 
ensures that the value of each padding bit is / and that the padding thus only have 
a permission. Storing a value in padding is a no-op (see Dehnition IB.351) . 

The side-condition ^unmapped (uJb) in the typing rule for a union uniont (*, w, b) 
of a specified variant ensures canonicity. Unions whose permissions are unmapped 
cannot be accessed and should therefore be in an unspecihed variant. This condition 
is essential for the separation algebra structure, see Section [721 
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Definition 6.26 Memories are defined as: 


m 6 mem := index ^fin (mtree x bool + type). 

Each object {w, /r) in memory is annotated with a Boolean /x to describe whether 
it has been allocated nsing malloc (in case /x = true) or as a block scope local, static, 
or global variable (in case /x = false). The types of deallocated objects are kept to 
ensnre that dangling pointers (which may remain to exist in memory, but cannot be 
used) have a unique type. 

Definition 6.27 The judgment T, A h m describes that the memory m is valid, ft 
is dehned as the conjunction of: 

1. For each o and t with mo = t we have: 

(a) A h o : T, (b) A o alive, and (c) T h t. 

2. For each o, w and /x with mo = (w, fi) we have: 

(a) A h o : T, (b) A h o alive, (c) T, A h w : t, and (d) not w all (0, /). 

The judgment A h o alive on object identifiers is defined in Definition 16.31 


Definition 6.28 The minimal memory typing environment m £ memenv of a mem¬ 
ory m is defined as: 


m ■.= Xo. 


(t, true) 

(typeof w, false) 


if m o = r 
if m o = {w, /x) 


Notation 6.29 We let T h m denote T, m h m. 


Many of the conditions of the judgment F, A h m ensure that the types of m 
match up with the types in the memory environment A (see Definition 16.31) . One 
may of course wonder why do we not define the judgment T h m directly, and even 
consider typing of a memory in an arbitrary memory environment. Consider: 

int X = 10, *p = &x; 

Using an assertion of separation logic we can describe the memory induced by 
the above program as x i—> 10 p &x. The separation conjunction * describes that 
the memory can be subdivided into two parts, a part for x and another part for p. 
When considering p &x in isolation, which is common in separation logic, we have 
a pointer that refers outside the part itself. This isolated part is thus not typeable by 
r h m, but it is typeable in the context of a the memory environment corresponding 
to the whole memory. See also Lemma [7.261 

In the remaining part of this section we will define various auxiliary operations 
that will be used to define the memory operations in Section [67^ We give a summary 


of the most important auxiliary operations: 



neWp : type —> mtree 

for 

7 : perm 

(_)[_]r : mem —> addr —> option mtree 



(_)[_//]r : mem —> addr —> mem 

for 

/ : mtree —)■ mtree 


Intuitively these are just basic tree operations, but unions make their actual 
definitions more complicated. The indeterminate memory tree neWp t consists of 
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indeterminate bits with permission 7 , the lookup operation m[a]r yields the memory 
tree at address a in m, and the alter operation m[a/f]-r applies the function / to 
the memory tree at address a in m. 

The main delicacy of all of these operations is that we sometimes have to interpret 
bits as memory trees, or reinterpret memory trees as memory trees of a different type. 
Most notably, reinterpretation is needed when type-punning is performed: 

union int_or_short { int x; short y; ]-u = { .x = 3]-; 
short z = u.y; 

This code will reinterpret the bit representation of a memory tree representing 
an int value 3 as a memory tree of type short. Likewise: 

union int_or_short { int x; short y; } u; 

((unsigned char*)&u)[0] = 3; 

((unsigned char*)&u) [1] = 0; 
short z = u.y; 

Here, we poke some bytes into the object representation of u, and interpret these 
as a memory tree of type short. 

We have dehned the flatten operation w that takes a memory tree w and yields 
its bit representation already in Definition l().24l We now define the operation which 
goes in opposite direction, called the unflatten operation. 

Definition 6.30 The unflatten operation (_)f : list pbit — > mtree is dehned as: 

(b)p := base^^ b 

(b)j;["! := array.^ ((b[o,s))r • ■ • (b[(n-i)s,ns))f) where s := bitsizeofr r 
/(b[o.so))r b[^o.. 2 o) 

(b)^truct t _ 

V(b[z„_i,z„_i+s„_i))r z„) 

where Ft = t, n := |t|, Si := bitsizeofr T~i and Zi := bitoffsetofr t i 
(b)“"'°" * := b 

Here, the operation (_)^ : pbit —> pbit is dehned as (x, b)^ := (x, /). 

Now, the need for uniorit b memory trees becomes clear. While unhattening a bit 
sequence as a union, there is no way of knowing which variant of the union the bits 
constitute. The operations (_) and (_)r are neither left nor right inverses: 

— We do not have (w)p = w for each w with T, A h iti : t. Variants of unions are 
destroyed by battening w. 

— We do not have (b)f = b for each b with |b| = bitsizeofr r either. Padding bits 
become indeterminate due to (_)^ by unhattening. 

In Section [72] we prove weaker variants of these cancellation properties that are 
sufficient for proofs about program transformations. 
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Definition 6.31 Given a permission 7 6 perm, the operation neWp : type —^ mtree 
that yields the indeterminate memory tree is defined as: 

new?r := (( 7 , /)b4sizeofr ^ 

The memory tree neWp t that consists of indeterminate bits with permission 
7 is used for objects with indeterminate value. We have dehned neWp r in terms 
of the unflattening operation for simplicity’s sake. This definition enjoys desirable 
structural properties such as neWp (r[n]) = (neWp r)". 

We will now define the lookup operation m[a]r that yields the subtree at address 
a in the memory m. The lookup function is partial, it will fail in case a is end-of-array 
or violates effective types. We hrst dehne the counterpart of lookup on memory trees 
and then lift it to memories. 


Definition 6.32 The lookup operation on memory trees (_)[_]r : mtree ref 
option mtree is dehned as: 




(array.^u;)[(-^ i)r|r := Wi[? 1 r 
(structt wb)[('- > i)i^T '■= Wi[r]r 


uniont {j,w,h))[{‘ -i)r|r := 

f wRr 

if i = j 

1 ((wb)[o,,))p[f]r 

if i j, q = •, exclusive (uJb) 

1 


if i ^ j, qr = 0 


where Tt = f and s = bitsizeofr "R 

(uniontb)[( -“"'°" f)f]r := (b[o. bitsizeofr r^))? [^r if T t = r, exclusive b 

The lookup operation uses the annotations q € {o, •} on ‘- >g i to give a formal 

semantics to the strict-aliasing restrictions [m 6.5.2.3]. 

— The annotation q = • allows a union to be accessed via a reference whose variant 
is unequal to the current one. This is called type-punning. 

— The annotation q = o allows a union to be accessed only via a reference whose 
variant is equal to the current one. This means, it rules out type-punning. 

Failure of type-punning is captured by partiality of the lookup operation. The 
behavior of type-punning of uniorit {j,w,h) via a reference to variant i is described 
by the conversion ((wbj^Q bitsizeofr T-i))r- '^'^6 memory tree w is converted into bits 
and reinterpreted as a memory tree of type Ti. 


Definition 6.33 The lookup operation on memories (_)[_]r : mem —> addr ^ 
option mtree is dehned as: 


m[a]r 


((u;[refra]r)[,,))— 
ff;[refr a]r 


if a is a byte address 
if a is not a byte address 


provided that m (index a) = (w, p). In omitted cases the result is T. In this dehnition 
we let i := bytep a ■ char_bits and j := (bytCp a -|- 1 ) ■ char_bits. 
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We have to take special care of addresses that refer to individual bytes rather 
than whole objects. Consider: 

struct S { int x ; int y;}'S = {.x = l, .y = 2]-; 

unsigned char z = ((unsigned char*)&s)[0]; 

In this code, we obtain the first byte ((unsigned char*)&s) [0] of the struct s. 
This is formalized by flattening the entire memory tree of the struct s, and selecting 
the appropriate byte. 

The Cll standard’s description of effective types EH 6.5p6-7] states that an 
access (which is either a read or store) affects the effective type of the accessed 
object. This means that although reading from memory does not affect the memory 
contents, it may still affect the effective types. Let us consider an example where it 
is indeed the case that effective types are affected by a read: 

short g(int *p, short *q) { 

short z = *q; *p = 10; return z; 

} 

int mainO { 

union int_or_short { int x; short y; ]■ u; 

// initialize u with zeros, the variant of u remains unspecified 

for (size_t i = 0; i < sizeof(u); i++) ((unsigned char*)&u) [i] = 0; 

return g(&u.x, feu.y); 

} 

In this code, the variant of the union u is initially unspecihed. The read *q in g 
forces its variant to x, making the assignment *p to variant y undefined. Note that 
it is important that we also assign undefined behavior to this example, a compiler 
may assume p and q to not alias regardless of how g is called. 

We factor these side-effects out using a function forcer : addr —^ mem —^ mem 
that updates the effective types (that is the variants of unions) after a successful 
lookup. The forcep function, as defined in Dehnition 1(1.51 can be described in terms 
of the alter operation 77i[a//]r that applies the function / : mtree — > mtree to the 
object at address a in the memory m and update variants of unions accordingly to 
a. To define forcer we let / be the identify. 


Definition 6.34 Given a function / : mtree —> mtree, the alter operation on memory 
trees (_)[_//]r : mtree —> ref —> mtree is defined as: 


■= fw 


(array.^u;)[(' —s- j)r//]r := array.^ {w[i := u;i[f7/]r]) 

, Struct t 


(uniont (i, w,b))[(-^^^^, j)r/f]r ■= 


(structt wb)[('- i)ry/]r := struck ((wb)[j := Wi[f/f]rhi]) 

uniont (i,w[r//]r,b) if * = 

uniont (j, (((wb)[o.s))p)[r//]r, (wb)[^_^j) if i A 

(union* b)[(=^^^5 i)f/f]r ■= union* (i, ((b[o,.))r')[r//]r,b[^,^)) 

In the last two cases we have T t = f, s := bitsizeofp t* and z := bitsizeofp (union t). 
The result of w[f/f]r is only well-dehned in case r(;[r)r A -L. 
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Definition 6.35 Given a function / : mtree —^ mtree, the alter operation on mem¬ 
ories (_)[_//]r : mem —> addr —> mem is defined as: 


m[a//]r 


m[(index a) := (?i)[refr a//]r, m)] if a is a byte address 
?7i[(index a) := (r(;[refr a//]r, /r)] if a is not a byte address 


provided that m (index a) = (w, p). In this definition we let: 


p /— p /— \ unsigned char — \typeof w 

J ^ J j))T ^[_7, bitsizeofr (typeof 

where i := bytep a ■ char_bits and j := (bytep a + 1) • char_bits. 

The lookup and alter operation enjoy various properties; they preserve typing 
and satisfy laws about their interaction. We list some for illustration. 

Lemma 6.36 (Alter commutes) // T, A h m, ai -Lp a 2 with: 

— r, A h ai : Ti, 7Ti[ai]p = wi, and T, A h /i wi : ti, and 

- r, A h 02 : T 2 , m[a 2 ]r = W 2 , and T, A h /2 W 2 : T 2 , 

then we have: 

"i[a2//2]r[ai//i]r = ?T^[ai//i]r[a2//2]r- 
Lemma 6.37 // T, A h m, m[o]p = w, and a is not a byte address, then: 

(m[a//]r)[a]r = fw. 

A variant of Lemma r6.37l for byte addresses is more subtle because a byte address 
can be used to modify padding. Since modifications of padding are masked, a suc¬ 
cessive lookup may yield a memory tree with more indeterminate bits. In Section [72] 
we present an alternative lemma that covers this situation. 

We conclude this section with a useful helper function that zips a memory tree 
and a list. It is used in for example Definitions 16.581 and 17.221 


Definition 6.38 Given a function / : pbit B ^ pbit, the operation that zips the 
leaves / : mtree — > list S ^ mtree is defined as: 

/ (base^, b) y := base^, (/by) 

/(array.^ w) y := array { f wq yp, «„).../w„-i y[s„_i.s„)) 
where n := |w| and Si := 

— ( / ^0 y[0, So) / 1^0 y[so,zo) 

f (structt wh) y := structt ... 

\f —1 y[zn-i, 2„_i-t-S.i_i) / b^—l y[zn-l+Sn-l, ^ 

where n := Si := |w7|, and Zi := bi| 

/(uniont (i,w,b)) y := uniorit {ijw yp, H),/b yj|_|_|_g|j) 


/ (uniont b) y := uniont (/b y) 
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6.4 Representation of values 

Memory trees (Definition l6.23ll are still rather low-level and expose permissions and 
implementation specific properties such as bit representations. In this section we 
define abstract values, which are like memory trees but have mathematical integers 
and pointers instead of bit representations as leaves. Abstract values are used in the 
external interface of the memory model. 

Definition 6.39 Base values are inductively dehned as: 

Ub G baseval ::= indetTb | nothing | in^ x \ ptrp | byte 6. 

While performing byte-wise operations (for example, byte-wise copying a struct 
containing pointer values), abstraction is broken, and pointer fragment bits have to 
reside outside of memory. The value byte b is used for this purpose. 


Definition 6.40 The judgment T, A hb Ub : ^b describes that the base value Ub has 
base type Tb. It is inductively defined as: 

r bb Tb Tb A void X : T\ 

r,A hb indetTb : r,A hb nothing : void r,A hb int^-ia: : t \ 

r, A h* p : cTp r, A h 6 1^1= char_bits Not b all in {0,1} Not b all / 

r, A hb ptrp : (Tp* ^ byte b : unsigned char 

The side-conditions of the typing rule for byte b ensure canonicity of representa¬ 
tions of base values. It ensures that the construct byte b is only used if b cannot be 
represented as an integer intunsigned char x or indet (unsigned char). 

In Definition 16.441 we dehne abstract values by extending base values with con¬ 
structs for arrays, structs and unions. In order to define the operations to look up 
and store values in memory, we dehne conversion operations between abstract val¬ 
ues and memory trees. Recall that the leaves of memory trees, which represent base 
values, are just sequences of bits. We therefore hrst dehne operations that convert 
base values to and from bits. These operations are called hatten and unhatten. 

Definition 6.41 The flatten operation (_) : baseval ^ list bit is dehned as: 

indetTb r := n 

nothing r .=/bltsizeofr void 
■:-r 

int^-, X \= X \ T] 

ptrp . (ptl* I P |o)o ■ ■ ■ I P lo)bitsizeofr (typeof p*) — l 

- —r 

byte 6 :=b 


The operation _ : rj : Z ^ list bool is defined in Definition 14.41 
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Definition 6.42 The unflatten operation (_)^ : list bit baseval is defined as: 
nothing 

{ intT-i (/3)T-i if & is a {0,1} sequence j3 
byte& if t, = unsigned char, not b all in {0,1}, and not b all / 
indetTi otherwise 

I ptrp if & = (ptr p)o • ■ ■ (ptr p)bitsizeofr and typeof p = ap 

|indet((Tp*) otherwise 

The operation (_)T-i : list bool —> Z is defined in Definition 14.41 

The encoding of pointers is an important aspect of the flatten operation related to 
our treatment of effective types. Pointers are encoded as sequences of frozen pointer 
fragment bits (ptr |p|o)i (see Definition 16.51 for the definition of frozen pointers). 
Recall that the flatten operation is used to store base values in memory, whereas the 
unflatten operation is used to retrieve them. This means that whenever a pointer p 
is stored and read back, the frozen variant \ p\o is obtained. 

Lemma 6.43 For each T, A hb r’b : we have (I'b”'")r = I |o- 

Freezing formally describes the situations in which type-punning is allowed since 
a frozen pointer cannot be used to access a union of another variant than its current 
one (Definition 16.321) . Let us consider an example: 

union U { int x; short y; ]-u = { .x = 3]-; 

short *p = &u.y; // a frozen version of the pointer &u.y is stored 
printf ( "°/,d" , *p) ; // type-punning via a frozen pointer -> undefined 

Here, an attempt to type-punning is performed via the frozen pointer p, which 
is formally represented as: 

, . union U , 

(o^ . union U, >-o l?0)signed short>*signed short* 

The lookup operation on memory trees (which will be used to obtain the value of 
*p from memory, see Definitions 16.321 and 16.58|) will fail. The annotation o prevents 
a union from being accessed through an address to another variant than its current 
one. In the example below type-punning is allowed: 

union U { int x; short y; ]-u = { .x = 3]-; 
printf ("°/,d", u.y); 

Here, type-punning is allowed because it is performed directly via u.y, which has 
not been stored in memory, and thus has not been frozen. 

Definition 6.44 Abstract values are inductively defined as: 

6 val ::= Ub I array.,, v \ structt v \ uniorit (i, v) | unions v. 

The abstract value unions-i; represents a union whose variant is unspecified. The 
values V correspond to interpretations of all variants of union t. Consider: 


{by/'’^ ■= 

m ■■= 

{b)r := 
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union U { int x; short y; int *p; } u; 

for (size_t i = 0; i < sizeof(u); i++) ((unsigned char*)&u)[i] = 0; 

Here, the object representation of u is initialized with zeros, and its variant thus 
remains unspecified. The abstract value of u i^: 

unionu[intsignedintO, intsigned short 0, indet (signed int=t) ] 

Recall that the variants of a union occupy a single memory area, so the sequence 
n of a union value uniont v cannot be arbitrary. There should be a common bit 
sequence representing it. This is not the case in: 

unionu [ intsigned int 0, intsigned short 1, indet (signed int=t) ] 

The typing judgment for abstract values guarantees that v can be represented 
by a common bit sequence. In order to express this property, we first define the 
unflatten operation that converts a bit sequence into an abstract value. 


Definition 6.45 The unflatten operation (_)p : list bit —> val is dehned as: 

(6)p := (&)p (the right hand side is Dehnition 16.411 on base values) 

(b)j;l"l := array^ ((6[o.s))f • ■ • (&[(n-i)s,ns))f) where s := bitsizeofpr 
(6)f-“ := struct* ((&[0,so))? • ■ • 

where F t = t, n := |r|, s* := bitsizeofy r* and Zi := bitoffsetofp t i 

(&)r" ‘ := ((Vso))r“ • • ■ (Vs„_o)r"-^) 

where F t = t, n := |t| and s* := bitsizeofp t* 


Definition 6.46 The judgment T, A h n : t describes that the value v has type t. 
It is inductively defined as: 


r, A hb Vb : "Tb 
r, A h Pb : Tb 

F t = T r, Al-i;:r 
r, A h struct* V : struct t 


r, A h i; : r |i;| = n A 0 
r, A h array^ v : r[n] 

F t = T i < \t\ T, a h p : r* 
r, A h union* (*, v) : union t 


Ft = T F,AFv-.t F,AFb Vi. (w* = (6[o, bitsizeofr r*))?) 
r, A h union* i; : union t 


The flatten operation (_) : val —> list bit, which converts an abstract value v into 

a bit representation irr, is more difficult to define (we need this operation to dehne 
the conversion operation from abstract values into memory trees, see Dehnition l6.49D . 
Since padding bits are not present in abstract values, we have to insert these. Also, 
in order to obtain the bit representation of an unspecified union* v value, we have to 
construct the common bit sequence b representing v. The typing judgment guarantees 
that such a sequence exists, but since it is not explicit in the value union* w, we have 
to reconstruct it from v. Consider: 


® Note that the Cll standard does not guarantee that the NULL pointer is represented as 
zeros, thus u.p is not necessarily NULL. 
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union U { struct S { short y; void *p; ]■ xl; int x2; 


Assuming sizeofp (signed int) = sizeofr (any=i=) = 4 and sizeofr (signed short) = 2, 
a well-typed union U value of an unspecified variant may be: 


V = unionu [structg [intsigped short 0, ptrp], int^igned intO]. 


The flattened versions of the variants of v are: 

structg [ intsigned short 0, ptrp] ’" = 0...00...0/./ (ptr p)o... (ptr p) 3 i 
i ntsigned int o'" = 0...00...00...00...0 

irr = 0...00...00...00...0 (ptr p)o ■ ■ ■ (ptr p) 3 i 

This example already illustrates that so as to obtain the common bit sequence 
irr of V we have to insert padding bits and “join” the padded bit representations. 


Definition 6.47 The jom operation on bits U : bit —>■ bit —>• bit is defined as: 

/□&:=& bU I ■= b bub ■.= b. 

Definition 6.48 The flatten operation (_) : val list bit is defined as: 

;= 

array := uT'" • ■ ■ U|iT|-i ^ 
structtu'" := (t¥T/°°)[o,^„) • • ■ 

where Ft = t, n := |t|, and Zi := bitoffsetofr t i 
union* {t,v) ^ := {un^on t)) 

union* := (urr/°°)[o, bfeizeofr (union t)) 

The operation ofvalp : list perm val mtree, which converts a value v of type 
T into a memory tree ofvalp 7 u, is albeit technical fairly straightforward. In principle 

it is just a recursive definition that uses the flatten operation for base values Ub 

_ rr _ ^ 

and the flatten operation union* v for unions union* v of an unspecified variant. 

The technicality is that abstract values do not contain permissions, so we have 
to merge the given value with permissions. The sequence 7 with I 7 I = bitsizeofp r 
represents a flattened sequence of permissions. In the definition of the memory store 
m{a := v)r (see Definition l6.58l) . we convert v into the stored memory tree ofvalp flv 
where 7 constitutes the old permissions of the object at address a. 
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Definition 6.49 The operation ofvalp : list perm —> val —> mtree is defined as: 

ofvalp 7 (fb) := basetypeof 7^ where b := 
ofvair 7 (array^ v) := array^ (ofvalp 7[o. s) do • • • ofvalp 7[(n-i)s. ns) d„-i) 
where s := bitsizeofp t and n := |i7| 

/of''alr7[o,so)do 7[l„,.p) 
ofvalp 7 (structt v) := struct^ • ■ ■ 

yofvalr7[.„_i,.„_i+.„_i)i'n-i %„_,+s„_,.z^) 
where Ft = t, n := |t|, Si := bitsizeofr Ti 
and Zi := bitoffsetofp r i 

ofvair 7 (uniont (i, v)) := uniorit (i, ofvair 7[o. *) d, 7[1 bitsizeofr (union t))) 
where s := bitsizeofr (typeof v) 
ofvair 7 (uniorit d) := uniont 7& where b := uniont d ^ 

Converting a memory tree into a value is as expected: permissions are removed 
and unions are interpreted as values corresponding to each variant. 

Definition 6.50 The operation tovalp : mtree —f val is dehned as: 

tovair (basert jb) := (6)p 

tovair (array^ w) := array^ (tovalp wq ... tovalp td|t;i|_i) 
tovair (structt wh) := structt (tovalp Wq ... tovalp td|t3|_i) 
tovair (uniont (*, w, b)) := uniont (*, tovalp w) 
tovair (UHbFt jb) := (6)r"'“" * 

The function tovalp is an inverse of ofvair up to freezing of pointers. Freezing is 
intended, it makes indirect type-punning illegal. 


Lemma 6.51 Given F, A h n : t, and let ^ be a flattened sequence of permissions 
with I 7 I = bitsizeofr n, then we have: 

tovair (ofvair 7 d) = | d |o. 

The other direction does not hold because invalid bit representations will become 
indeterminate values. 

struct S { int *p; ]■ s; 

for (size_t i = 0; i < sizeof (s); i++) ((unsigned char*)&s)[i] = i; 

// s has some bit representation that does not constitute a pointer 
struct S s2 = s; 

// After reading s, and storing it, there are no guarantees about s2, 
// whose object representation thus consists of Is 

We finish this section by defining the indeterminate abstract value newp t, which 
consists of indeterminate base values. The definition is similar to its counterpart on 
memory trees fDehnition l6.31l) . 
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Definition 6.52 The operation newr : type —^ val that yields the indeterminate 
value is defined as: 

newrr := . 

Lemma 6.53 //T h r, then: 

tovair (neWp t) = newp r and ofvair (newp r) = neWp t. 

6.5 Memory operations 

Now that we have all primitive definitions in place, we can compose these to imple¬ 
ment the actual memory operations as described in the beginning of this section. 
The last part that is missing is a data structure to keep track of objects that have 
been locked. Intuitively, this data structure should represent a set of addresses, but 
up to overlapping addresses. 

Definition 6.54 Locksets are dehned as: 

e lockset := Pfin(index x N). 

Elements of locksets are pairs (o, i) where o e index describes the object identifier 
and i 6 N a bit-offset in the object described by o. We introduce a typing judgment 
to describe that the structure of locksets matches up with the memory layout. 

Definition 6.55 The judgment T, A h £1 describes that the loekset is valid, ft is 
inductively defined as: 

for each (o, f) £ £i there is a t with A h o : t and i < bitsizeofr t 

r, A h £i 

Definition 6.56 The singleton loekset {-}r : addr lockset is defined as: 

{a}r := {(index a, i) \ bitoffsetp a < i < bitoffsetr a -|- bitsizeofr (typeof a)}. 
Lemma 6.57 J/T, A h Oi : oi and T, A h 02 : 02 and T h { 01 , 02 } strict, then: 

Oi J-r 0,2 implies {ai}r H { 02 }r = 0 - 
Definition 6.58 The memory operations are defined as: 

m{a)r ■= tovalp w if m[a]r = w and Vi . Readable C kind (w)i 
forcer a m := m[(index a) := (w[refr a/\w' . w^]r, m)] if ™ (index a) = {w, /i) 
m{a := v)r ■= m[a/Xw . ofvair (wi) f]r 
writabler a m := . m[o]r = w and Vi . Writable C kind (w)i 

lockr a m := m[a/Xw . apply lock to all permissions of w]r 
unlock 0, m := {(o, {f w y, /i)) | mo = {w, /i)} U {(o, t) | mo = r} 
where f ( 7 , b) true := (unlock 7 , b) 

/( 7 , 6 ) false := ( 7 , 5), 

and y := ((o, 0 ) £ fl)... ((o, |bitsizeofr (typeof w)\ — 1 ) £ fl) 
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allocr ov 11 m:=m[o := (ofvair (0(0, Oyp^f «)) ^)] 

T[n] 

freeable a m := 3ot a nw . a = (o:t, '—> 0, 0)t>,o-, mo = (w, true) 

and all w have the permission 0(0, 1) 

free o m ■.= m[o := typeof re] if mo = (le, /r) 

The lookup operation m{a)r uses the lookup operation m[a]r that yields a mem¬ 
ory tree w fDefinition l6.3dll . and then converts w into the value tovair w. The opera¬ 
tion m[a]r already yields T in case effective types are violated or a is an end-of-array 
address. The additional condition of m{a)r ensures that the permissions allow for a 
read access. Performing a lookup affects the effective types of the object at address a. 
This is factored out by the operation forcer « m which applies the identity function 
to the subobject at address a in the memory m. Importantly, this does not change 
the memory contents, but merely changes the variants of the involved unions. 

The store operation m{a := n)r uses the alter operation m[a/Xw .ofvair (wi) u]r 
on memories fDehnition l6.35l) to apply Xw . ofvair (wi) v to the subobject at address 
a. The stored value v is converted into a memory tree while retaining the permissions 
uJi of the previously stored memory tree w at address a. 

The dehnition of lockr a m is straightforward. In the Coq development we use 
a map operation on memory trees to apply the function lock fDefinition 15.51) to the 
permission of each bit of the memory tree at address a. 

The operation unlock Q m unlocks a whole lockset fl, rather than an individual 
address, in memory m. For each memory tree w at object identifier o, it converts fl to 
a Boolean vector y = ((o, 0) e fi)... ((o, jbitsizeofr (typeof w)| — 1) e fl) and merges 
w with y (using Dehnition 16.381) to apply unlock IDehnition 15.51) to the permissions 
of bits that should be unlocked in w. We show some lemmas to illustrate that the 
operations for locking and unlocking enjoy the intended behavior: 

Lemma 6.59 // T, A h m and T, A h a : r and writablep a m, then we have: 

locks (lockr a m) = locks m U {a}r- 

Lemma 6.60 If O, C locks m, then locks (unlock Q m) = locks m\Q,. 

Provided o i. dom m, allocation allocr o v /i m extends the memory with a new 
object holding the value v and full permissions 0(0, 1). Typically we use v = newr t 
for some t, but global and static variables are allocated with a specihc value v. 

The operation free o m deallocates the object o in m, and keeps track of the type 
of the deallocated object. In order to deallocate dynamically obtained memory via 
free, the side-condition freeable a m describes that the permissions are sufficient for 
deallocation, and that a points to the first element of an malloced array. 

All operations preserve typing and satisfy the expected laws about their interac¬ 
tion. We list some for illustration. 

Fact 6.61 If writabler a m, then there exists a value v with a(m)r = v. 

Lemma 6.62 (Stores commute) // T, A h m and oi Tr U 2 with: 

— r, A P Oi : Ti, writabler Ui m, and T, A h Ui : ti, and 

— r, A h 02 : T 2 , writabler 0-2 m, and T, A h U 2 : T 2 , 
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then we have: 


m{a2 ■■= V2)r{ai '■= Vi)r = rn{ai := Vi)r{a2 := U2)r- 

Lemma 6.63 (Looking up after storing) If F, A h m and F, A h a : t and 

r, A h : r and writabler a m and a is not a byte address, then we have: 

{m{a := v)r)(a)r = |v|o- 

Storing a value v in memory and then retrieving it, does not necessarily yield the 
same value v. It intentionally yields the value | v |o whose pointers have been frozen. 
Note that the above result does not hold for byte addresses, which may store a value 
in a padding byte, in which case the resulting value is indeterminate. 

Lemma 6.64 (Stores and look ups commute) IfF,Ahm and ai _Lr “2 cmd 
r, A h 02 : r2 and writabler 0-2 rn and F, A h t;2 : 72, then we have: 

m{ai)r=Vi implies {m{a 2 '.= V 2 )r){ai)r = Vi. 

These results follow from Lemma 16.36116.371 and 16.511 


7 Formal proofs 

7.1 Type-based alias analysis 

The purpose of Oil’s notion of effective types EZl 6 .5p6-7] is to make it possible for 
compilers to perform typed-based alias analysis. Consider: 

short g(int *p, short *q) { 

short X = *q; *p = 10; return x; 

} 

Here, a compiler should be able to assume that p and q are not aliased because 
they point to objects with different types (although the integer types signed short 
and signed int may have the same representation, they have different integer ranks, 
see Dehnition EH and are thus different types). If g is called with aliased pointers, 
execution of the function body should have undefined behavior in order to allow a 
compiler to soundly assume that p and q are not aliased. 

From the Oil standard’s description of effective types it is not immediate that 
calling g with aliased pointers results in undefined behavior. We prove an abstract 
property of our memory model that shows that this is indeed a consequence, and that 
indicates a compiler can perform type-based alias analysis. This also shows that our 
interpretation of effective types of the Oil standard, in line with the interpretation 
from the GCC documentation m, is sensible. 

Definition 7.1 A type r is a subobject type of a, notation r Cp a, if there exists 
some reference r with F h r : cr ^ r. 

For example, int [2] is a subobject type of struct S { int x [2] ; int y [3] ; } 
and int [2] [2], but not of struct S { short x[2] ; }, nor of int(*) [2]. 
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Theorem 7.2 (Strict-aliasing) Given F, A h m, frozen addresses ai and 02 with 
A, m h ai : (7i and A,7Tt h a 2 : cf 2 and ai,a 2 A unsigned char, then either: 

1. We have cr\ Cp 0-2 or U 2 Cp 

2. We have Oi Ap 02 - 

3. Accessing ai after accessing 02 and vice versa fails. That means: 

(a) (forcep a 2 m)(ai)p = A and (forcep ai m){a 2 )r = A, and 

(h) m{a 2 '■= fi)r(Hi)r = A and m{a\ := W 2 )r(o 2 )r = A for all stored values Vi 
and 1 ) 2 . 

This theorem implies that accesses to addresses of disjoint type are either non¬ 
overlapping or have undefined behavior. Fact 16.61] accounts for a store after a lookup. 
Using this theorem, a compiler can optimize the generated code in the example based 
on the assumption that p and q are not aliased. Reconsider: 

short g(int *p, short *q) { short x = *q; *p = 10; return x; ]- 

If p and q are aliased, then calling g yields undehned behavior because the assign¬ 
ment *p = 10 violates effective types. Let m be the initial memory while executing 
g, and let Op and a^ be the addresses corresponding to p and q, then the condition 
writablep Op (forcep Oq m) does not hold by Theorem 17.21 and Fact 16.611 


7.2 Memory rehnements 

This section defines the notion of memory refinements that allows us to relate mem¬ 
ory states. The author’s PhD thesis [331 shows that the CH 2 O operational semantics 
is invariant under this notion. Memory refinements form a general way to validate 
many common-sense properties of the memory model in a formal way. For example, 
they show that the memory is invariant under relabeling. More interestingly, they 
show that symbolic information (such as variants of unions) cannot be observed. 

Memory refinements also open to door to reason about program transformations. 
We demonstrate their usage by proving soundness of constant propagation and by 
verifying an abstract version of memcpy. 

Memory refinements are a variant of Leroy and Blazy’s notion of memory exten¬ 
sions and injections EH- A memory refinement is a relation mi Cp m 2 between a 
source memory state mi and target memory state m 2 , where: 

1. The function / : index — ^ option (index x ref) is used to rename object identifiers 
and to coalesce multiple objects into subobjects of a compound object. 

2. Deallocated objects in mi may be replaced by arbitrary objects in m 2 . 

3. Indeterminate bits / in mi may be replaced by arbitrary bits in m 2 . 

4. Pointer fragment bits {ptr p)i that belong to deallocated pointers in mi may be 
replaced by arbitrary bits in m 2 . 

5. Effective types may be weakened. That means, unions with a specific variant in 
mi may be replaced by unions with an unspecified variant in m 2 , and pointers 
with frozen union annotations o in mi may be replaced by pointers with unfrozen 
union annotations • in m 2 . 

The key property of a memory refinement mi m 2 , as well as of Leroy and 
Blazy’s memory extensions and injections, is that memory operations are more de¬ 
fined on the target memory m 2 than on the source memory mi. For example, if a 
lookup succeeds on mi, it also succeed on m 2 and yield a related value. 
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The main judgment mi q£ memory refinements will be built using 

a series of refinement relations on the structures out of which the memory consists 
(addresses, pointers, bits, memory trees, values). All of these judgments should satisfy 
some basic properties, which are captured by the judgment Ai A2. 

Definition 7.3 A renaming function f : index —> option (index x ref) is a refinement, 
notation Ai A2, if the following conditions hold: 

1 . If / Oi = (o, ri) and f 02 = (o, fjj), then oi = 02 or fi ± {injectivity). 

2 . If / oi = (02, f), then frozen f. 

3 . If / oi = (02, f) and Ai h oi : cr, then A2 h 02 : t and T h r : r >-> a for a r. 

4 . If / oi = (02, f) and A2 b 02 : t, then Ai h oi : cr and T h r : r >-> a for a cr. 

5 . If / oi = (02, f) and Ai h oi alive, then A2 b 02 alive. 

The renaming function / : index —^ option (indexx ref) is the core of all refinement 

judgments. It is used to rename object identifiers and to coalesce multiple source 
objects into subobjects of a single compound target object. 

Struct t struct t 

Consider a renaming / with j oi = (oi, ^- > 0 ) and j 02 = (oi, ^-)■ 1 ), and 

an environment T with Ft = [Ti,r2]. This gives rise to following refinement: 



Injectivity of renaming functions guarantees that distinct source objects are co¬ 
alesced into disjoint target subobjects. In the case of Blazy and Leroy, the renaming 
functions have type index —> option (index x N), but we replaced the natural number 
by a reference since our memory model is structured using trees. 

Since memory refinements rearrange the memory layout, addresses should be 
rearranged accordingly. The judgment ai describes how 02 is 

obtained by renaming ui according to the renaming /, and moreover allows frozen 
union annotations o in Ui to be changed into unfrozen ones • in 02. The index rp in 
the judgment ai corresponds to the type of Oi and 02. 

The judgment for addresses is lifted to the judgment for pointers in the obvious 
way. The judgment for bits is inductively defined as: 

P e { 0 ,1} Pi frozen p2 i < bitsizeofr (up’r) 

P P (ptrpi)i CpAiH^Aa (ptrp 2 )i 

r, A 2 b 6 r, Ai b a : (T Ai F a alive T, A 2 b 6 

/ cTA.^A 2 ^ (ptr a), cbA,^A2 ^ 

The last two rules allow indeterminate bits /, as well as pointer fragment bits 
(ptr a)i belonging to deallocated storage, to be replaced by arbitrary bits b. 
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The judgment is lifted to memory trees following the tree structure and using 
the following additional rule: 

r t = T r, A h Wi : Ti wTbi ^ 

bitsizeofp (union t) = bitsizeofr ti + |bi| -lunmapped (uJi bi) 
uniont (i,wi,bi) un|ontb2 : union t 

This rule allows a union that has a specihc variant in the source to be replaced 
by a union with an unspecified variant in the target. The direction seems counter 
intuitive, but keep in mind that unions with an unspecified variant allow more be¬ 
haviors. 

Lemma 7.4 If wi T, Ai h wi : r and T, A2 h W2 : r. 

This lemma is useful because it removes the need for simultaneous inductions on 
both typing and refinement judgments. 

We define TTii Cp m2 as mi where the judgment mi 

is defined such that if / Oi = (02, r), then: 



The above definition makes sure that objects are renamed, and possibly coalesced 
into subobjects of a compound object, as described by the renaming function /. 

In order to reason about program transformations modularly, we show that mem¬ 
ory refinements can be composed. 


Lemma 7.5 Memory refinements are reflexive for valid memories, that means, if 
r, A h m, then m c^:Ai->a ^ inhere id o := (o, e). 

Lemma 7.6 Memory refinements compose, that means, if mi □/■Ai'->A 2 
m2 ■A2 i->A3 tfien mi °/.Aii->A3 where: 

, t, J(o 3 , if/oi = (02, r2)and/'o2 = (03, fa) 

^ I _L otherwise 


All memory operations are preserved by memory refinements. This property is 
not only useful for reasoning about program transformations, but also indicates that 
the memory interface does not expose internal details (such as variants of unions) 
that are unavailable in the memory of a (concrete) machine. 

Lemma 7.7 If mi □/■Ai'->A2 q/.Aii-iA2 ^ mi(ai)r = vi, then 

there exists a value V2 with m2{a2)r = V2 and Vi 

Lemma 7.8 If mi □/■Ai'->A2 ^/.Aih-iA2 ^ ^/.Aii->A2 ^ 

and writablepmi ai, then: 
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1. We/lowe writabler m 2 a 2 - 

2. We have mi{a\ := Vi)r m 2 {a 2 := V 2 )r- 

As shown in Lemma [6.631 storing a value v in memory and then retrieving it, 
does not necessarily yield the same value v. In case of a byte address, the value may 
have been stored in padding and therefore have become indeterminate. Secondly, 
it intentionally yields the value | n |o in which all pointers are frozen. However, the 
widely used compiler optimization of constant propagation, which substitutes values 
of known constants at compile time, is still valid in our memory model. 

Lemma 7.9 // F, A h w : t, then \ v |o 

Theorem 7.10 (Constant propagation) //r,A h m and r,A h o : t and 

r, A h n : r and writablep a m, then there exists a value v' with: 

m(a := v}r{a}r = v' and v' Cp v : t. 

Copying an object w by an assignment results in it being converted to a value 
tovalp w and back. This conversion makes invalid representations of base values in¬ 
determinate. Copying an object w byte-wise results in it being converted to bits w 
and back. This conversion makes all variants of unions unspecihed. The following 
theorem shows that a copy by assignment can be transformed into a byte-wise copy. 

Theorem 7.11 (Memcpy) J/F, A h w : r, then: 

ofvalp (wi) (tovalp w) Cp w Hp (w)p : t. 

Unused reads cannot be removed unconditionally in the CH 2 O memory model 
because these have side-effects in the form of uses of the forcep operation that updates 
effective types. We show that uses of forcep can be removed for frozen addresses. 

Theorem 7.12 // F, A h m and m{a)r ^ -L and frozen a, then forcep a m Cp m. 


7.3 Reasoning about disjointness 

In order to prove soundness of the CH 2 O axiomatic semantics, we were often in need 
to to reason about preservation of disjointness under memory operations [S^. This 
section describes some machinery to ease reasoning about disjointness. We show that 
our machinery, as originally developed in ISD, extends to any separation algebra. 

Definition 7.13 Disjointness of a list x, notation _Lx, is defined as: 

1. _L£ 

2. llDx and a: ± (J a;, then ± (a; x) 

Notice that -La; is stronger than having Xi -L Xj for each i + j. For example, 
using fractional permissions, we do not haveT [ 0.5, 0.5, 0.5] whereas 0.5 ± 0.5 clearly 
holds. Using disjointness of lists we can for example state the associativity law (law[2] 
of Definition 15.II) in a symmetric way: 


Fact 7.14 If ± (a: y z), then x U {y U z) = {x U y) U z. 
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We define a relation X\ =x X 2 that expresses that Xi and X 2 behave equivalently 
with respect to disjointness. 

Definition 7.15 Equivalence of lists Xi and X 2 with respect to disjointness, notation 
Xi =x X 2 , is defined as: 


Xi X 2 := Vx ._L (a; afi) — t-L (a; £ 2 ) 
ail =x a;2 := Xi <± X 2 A a;2 <± Xi 


It is straightforward to show that <x is reflexive and transitive, is respected by 
concatenation of lists, and is preserved by list containment. Hence, =± is an equiva¬ 
lence relation, a congruence with respect to concatenation of lists, and is preserved 
by permutations. The following results (on arbitrary separation algebras) allow us 
to reason algebraically about disjointness. 

Fact 7.16 If Xi <x X 2 , thenToii implies-L a ;2 • 

Fact 7.17 If Xi =± X 2 , then Tail iff_La; 2 - 

Theorem 7.18 We have the following algebraic properties: 


Xi U X 2 =± Xi X 2 provided that Xi _L a :2 

provided that _L x 

X 2 =± Xi (x 2 \ Xi) provided that a;i C X 2 



In Section O we show that we have similar properties as the above for the 
specific operations of our memory model. 

7.4 The memory as a separation algebra 

We show that the CH 2 O memory model is a separation algebra, and that the sep¬ 
aration algebra operations interact appropriately with the memory operations that 
we have defined in Sectional 

In order to define the separation algebra relations and operations on memories, 
we first define these on memory trees. Memory trees do not form a separation algebra 
themselves due to the absence of a unique 0 element (memory trees have a distinct 
identity element newf for each type r, see Definition 16.511) . The separation algebra 
of memories is then defined by lifting the definitions on memory trees to memories 
(which are basically finite functions to memory trees). 

Definition 7.19 The predicate valid : mtree —> Prop is inductively defined as: 


valid b valid w valid w valid b 


valid (base^ b) valid (array, w) 
valid w valid b -lunmapped (uJb) valid b 
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Fact 7.20 If r, A h w : r, then valid w. 

The valid predicate specihes the subset of memory trees on which the separation 
algebra structure is dehned. The definition basically lifts the valid predicate from the 
leaves to the trees. The side-condition ^unmapped (uJb) on uniorit (i,w,b) memory 
trees ensures canonicity, unions whose permissions are unmapped cannot be accessed 
and are thus kept in unspecihed variant. Unmapped unions unionj b can be combined 
with other unions using U. The rationale for doing so will become clear in the context 
of the separation logic in the author’s PhD thesis |33| . 


Definition 7.21 The relation _L : mtree mtree —> Prop is inductively defined as: 

bi _L b2 wi _L W 2 wi _L W2 bi _L b2 

baseT^ bi _L base^^ b2 array Wi _L array.^ W2 structt tcibi -L struck W2h2 

Wi J-11)2 bi-L 62 -lunmapped (uT bi) -lunmapped 62) 
uniorit (*, Wi, bi) _L unionj (i, u'2, 62) 

bi _L 62 uT bi-L 62 valid -lunmapped (uTf bi) unmapped 62 

uniont bi ± union^ 62 uniont (i, wi, bi) -L unionj b2 

bi ± wi'62 valid W2 unmapped bi -^unmapped (wi 62) 
uniont bi -L uniont (*, ^2, 62) 

Definition 7.22 The operation U : mtree ^ mtree —^ mtree is defined as: 


bascTT, bi U bascTi, ^>2 
array^ Wi U array^ W2 

structt wibi U structt W2b2 
uniont (*, Wi, bi) U uniont (*, u'2, b2) 
uniont bi U uniont bi 
uniont (*, Wi, bi) U uniont 62 

uniont bi U uniont (b u;2, b2) 


:= basCrt (^1 U 62) 

:= array.^ (rfii U W2) 

:= structt (wibi U W2b2) 

:= uniont (*, wi U W2, bi U b2) 
:= uniont (bi U 62) 

:= uniont (*,^1,61) U 62 
:= uniont (*,W2,b2) 0 bi 


In the last two clauses, w 0 b is a modified version of the memory tree w in which the 
elements on the leaves of w are zipped with b using the U operation on permission 
annotated bits (see Definitions 16.381 and 15.81) . 

The definitions of valid, _L and U on memory trees satisfy all laws of a separation 
algebra (see Definition [CT apart from those involving 0. We prove the cancellation 
law explicitly since it involves the aforementioned side-conditions on unions. 


Lemma 7.23 If -L Wi and W3 _L W 2 then: 


W3 U wi = Wi U W 2 implies wi = W2 . 
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Proof By induction on the derivations Ws -L and W3 _L ui2- We consider one case: 
uniorit (ijWsjba) ± unions (i, wi,bi) unions (i, 1^3,63) ± unions 62 
uniorit (*, W3, 63) U uniorit (*, wi, bi) = uniont (i, W3, 63) U uniont 62 
uniont (i, wi, bi) = uniont 62 

Here, we have W3 103 U wi bi = 63 U 62 by assumption, and therefore wf bi = 62 

by the cancellation law of a separation algebra. However, by assumption we also have 
-■unmapped (taf bi) and unmapped 62, which contradicts wi bi = 62- 

Definition 7.24 The separation algebra of memories is defined as: 

valid m :=\/ow .mo = (w, p) —t (valid w and not w all (0, /)) 

TTii _L m2 '.= yo. P mi m2 o 
mi U m2 '.= Xo . f mi m2 o 

P : mem ^ mem index Prop and / : mem —> mem —> index —> option mtree are 
defined by case analysis on mi o and m2 o: 


mi 0 

m2 0 

P mi m2 0 

/ mi m2 0 

(wi, p) 

(Wi, p) 

wi Ilw2, not wi all (0, /) and not W2 all (0, /) 

{Wl U W 2 , p) 

{wi, p) 

_L 

valid wi and not Wf all (0, /) 

{wi, p) 

_L 

{W 2 , p) 

valid W2 and not wf. all (0, /) 

{W 2 , p) 

n 

_L 

True 

n 

_L 

T 2 

True 

T 2 

_L 

_L 

True 

_L 

otherwise 

False 

_L 


The definitions of the omitted relations and operations are as expected. 

The emptiness conditions ensure canonicity. Objects that solely consist of in¬ 
determinate bits with 0 permission are meaningless and should not be kept at all. 
These conditions are needed for cancellativity. 

Fact 7.25 If F, A h m, then valid m. 

Lemma 7.26 If mi _L m2, then: 

r, A h mi U m2 iff r, A h mi and F, A h m2. 


Notice that the memory typing environment A is not subdivided among mi and 
m2. Consider the memory state corresponding to int x = 10, *p = &x: 



Here, w is the memory tree that represents the integer value 10 . The pointer on 
the right hand side is well-typed in the memory environment 1—f w, Op 1—^ • of the 
whole memory, but not in Op i-A •. 

We prove some essential properties about the interaction between the separation 
algebra operations and the memory operations. These properties will be used in the 
soundness proof of the separation logic in the author’s PhD thesis | 33 | . 
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Lemma 7.27 (Preservation of lookups) //F, A h mi and mi C m2, then: 

mi{a)r = V implies m2{a)r = v 

writablepami implies writableram2 

The relation C is part of a separation algebra, see Definition \ 5 . 1 l We have mi C m2 
iff there is an m3 with mi _L m3 and m2 = mi U m3 . 


Lemma 7.28 (Preservation of disjointness) // F, A h m then: 


m <x forcer a m 
m <±m{a := w)r 
m <xlockr a m 
m <xunlock Q m 


if F, A h a : T and m{a)r + -L 

if F, A h a : T and writablep a m 

if F, A h a : T and writablep a m 

if C locks m 


The relation <x is defined in Definition \ 7 . 15 \ If m <x 'm', then each memory that 
is disjoint to m is also disjoint to m'. 

As a corollary of the above lemma and Fact 17 .1(11 we obtain that mi _L m2 implies 
disjointness of the memory operations: 

forcer a -L 1^2 mi (a := n)r -L m2 

lockr a mi ± m2 unlock mi ± m2 


Lemma 7.29 (Unions distribute) J/F, A h m and mi _L m2 then: 


forcer a (mi U m2) = forcer a mi U m2 
(mi U m2){a := v)r = mi{a := i’)r U m2 
lockr a (mi U m2) = lockr a mi U m2 
unlock n (mi U m2) = unlock mi U m2 


if F, A h a : T and mi (a)r -L 
if F, A h {a, w} : r and writabler a mi 
if F, A h a : T and writabler cl mi 
if n C locks mi 


Memory trees and memories can be generalized to contain elements of an ar¬ 
bitrary separation algebra as leaves instead of just permission annotated bits | 32 | . 
These generalized memories form a functor that lifts the separation algebra structure 
on the leaves to entire trees. We have taken this approach in the Coq development, 
but for brevity’s sake, we have refrained from doing so in this paper. 


8 Formalization in Coq 

Real-world programming language have a large number of features that require large 
formal descriptions. As this paper has shown, the C programming language is not 
different in this regard. On top of that, the C semantics is very subtle due to an abun¬ 
dance of delicate corner cases. Designing a semantics for C and proving properties 
about such a semantics therefore inevitably requires computer support. 

For these reasons, we have used the Coq proof assistant m to formalize all 
definitions and theorems in this paper. Although Coq does not guarantee the absence 
of mistakes in our definitions, it provides a rigorous set of checks on our definitions. 
Already Coq’s type checking of definitions provides an effective sanity check. On top 
of that, we have used Coq to prove all metatheoretical results stated in this paper. 
Last but not least, using Coq’s program extraction facility we have extracted an 
exploration tool to test our memory model on small example programs Eng. 
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8.1 Overloaded typing judgments 

Type classes are used to overload notations for typing judgments (we have 25 different 
typing judgments). The class Valid is used for judgments without a type, such as 
h r and r, A h m. 

Class Valid (E A : Type) := valid; E —> A ^ Prop. 

Notation F }" := (valid T) . 

Notation F }*" := (Forall (■/ {F})). 

We use product types to represent judgments with multiple environments such 
as F, A h m. The notation -/ {F}* is used to lift the judgment to lists. The class 
Typed is used for judgments such as F, A h p : r and F, A, r h e : rir. 

Class Typed (E T V : Type) := typed: E V —> T ^ Prop. 

Notation "F h v : t" := (typed F v t). 

Notation "F h* vs :* rs" := (Forall 2 (typed F) vs ts). 


8.2 Implementation defined behavior 

Type classes are used to parametrize the whole Coq development by implementation 
defined parameters such as integer sizes. For example, Lemma [ 6.511 looks like: 

Lemma to_of_val ‘{EnvSpec K]- F A 7s v r : 

y F —^ (F,A) h V ; r —> length 7s = bit_size_of F t ^ 
to_val F (of_val F 7s v) = freeze true v. 

The parameter EnvSpec K is a type class describing an implementation environ¬ 
ment with ranks K (Definition 14 . 1211 . Just as in this paper, the type K of integer 
ranks is a parameter of the inductive definition of types (see Definition SH) and is 
propagated through all syntax. 

Inductive signedness := Signed I Unsigned. 

Inductive int_type (K: Set) := IntType {sign: signedness; reink: KJ. 

The definition of the type class EnvSpec is based on the approach of Spitters and 
van der Weegen approach [^. We have a separate class Env for the operations that 
is an implicit parameter of the whole class and all lemmas. 

Class Env (K: Set) := { 
env_type_env :> IntEnv K; 
size_of : env K ^ type K —>■ nat; 
align_of : env K type K —>■ nat; 
field_sizes ; env K —> list (type K) —> list nat 

}. 

Class EnvSpec (K: Set) ‘{Env KJ := { 
int_env_spec :» IntEnvSpec K; 
size_of_ptr_ne _0 F Tp : size_of F (vp.*) A 0 ; 
size_of_int F 7 : size_of F (intT 7) = rank_size (rank 7); 

}. 
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8.3 Partial functions 

Although many operations in CH 2 O are partial, we have formalized many such op¬ 
erations as total functions that assign an appropriate default value. We followed the 
approach presented in Section 15.21 where operations are combined with a validity 
predicate that describes in which case they may be used. For example, part (2) of 
Lemma [7.291 is stated in the Coq development as follows: 

Lemma mem_insert_union ‘{EnvSpec K} F A ml m2 al vl rl : 
y F —> y {r.A} mi ^ mi _L m2 ^ 

(r,Z\) h al : TType rl —> mem_writable F al ml (F.A) h vl : rl 
< [al: =vl]{r]->(ml U m2) = < [al: =vl] {r]->ml U m2. 

Here, ml _L m2 is the side-condition of ml U m2, and mem_writable F al ml 
the side-condition of < [al: =vl] {F}>ml. Alternatives approaches include using the 
option monad or dependent types, but our approach proved more convenient. In 
particular, since most validy predicates are given by an inductive definition, various 
proofs could be done by induction on the structure of the validy predicate. The cases 
one has to consider correspond exactly to the domain of the partial function. 

Admissible side-conditions, such as in the above example < [al: =vl] {F}>ml _L m2 
and mem_writable F al (ml U m2), do not have to be stated explicitly and follow 
from the side-conditions that are already there. By avoiding the need to state admis¬ 
sible side-conditions, we avoid a blow-up in the number of side-conditions of many 
lemmas. We thus reduce the proof effort needed to use such a lemma. 


8.4 Automation 

The proof style deployed in the CH 2 O development combines interactive proofs with 
automated proofs. In this section we describe some tactics and forms of proof au¬ 
tomation deployed in the CH 2 O development. 

Small inversions. Coq’s inversion tactic has two serious shortcomings on induc¬ 
tively defined predicates with many constructors. It is rather slow and its way of 
controlling of names for variables and hypotheses is deficient. Hence, we are often 
using the technique of small inversions by Monin and Shi [IS] that improves on both 
shortcomings. 

Solving disjointness. We have used Coq’s setoid machinery to enable rewriting 
using the relations <_l and =x (Definition 17.151) . Using this machinery, we have 
implemented a tactic that automatically solves entailments of the form: 

Hq . -L Xqj * ■ * 7 ^71 • 4- Xji —r b -Lx 

where x and Xi (for i < n) are arbitrary Coq expressions built from 0, U and IJ. 
This tactic works roughly as follows: 

1. Simplify hypotheses using Theorem 17.181 

2. Solve side-conditions by simplification using Theorem 17.181 and a solver for list 
containment (implemented by reflection). 
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3. Repeat these steps until no further simplification is possible. 

4. Finally, solve the goal by simplification using Theorem IT.isl and list containment. 

This tactic is not implemented using reflection, but that is something we intend 
to do in future work to improve its performance. 

First-order logic. Many side-conditions we have encountered involve simple entail- 
ments of first-order logic such as distributing logical quantifiers combined with some 
propositional reasoning. Coq does not provide a solver for first-order logic apart from 
the firstorder tactic whose performance is already insufficient on small goals. 

We have used Ltac to implemented an ad-hoc solver called naive_solver, which 
performs a simple breath-first search proof search. Although this tactic is inherently 
incomplete and suffers from some limitations, it turned out to be sufficient to solve 
many uninteresting side-conditions (without the need for classical axioms). 


8.5 Overview of the Coq development 

The Coq development of the memory model, which is entirely constructive and axiom 
free, consists of the following parts: 


Component 

Sections 

LOG 

Support library (lists, finite sets, finite maps, etc.) 

Section [2] 

12 524 

Types & Integers 

Section H] 

1928 

Permissions & separation algebras 

Section [S] 

1811 

Memory model 

Section [6] 

8 736 

Refinements 

Section 17.21 

4046 

Memory as separation algebra 

Section 17.41 

3 844 

Total 

32 889 


9 Related work 

The idea of using a memory model based on trees instead of arrays of plain bits, and 
the idea of using pointers based on paths instead of offsets, has already been used 
for object oriented languages. It goes back at least to Rossie and Friedman m, and 
has been used by Ramananandro et al. [15] for C-I--I-. Furthermore, many researchers 
have considered connections between unstructured and structured views of data in 
C [55l[mi2l[2T] in the context of program logics. 

However, a memory model that combines an abstract tree based structure with 
low-level object representations in terms of bytes has not been explored before. In 
this section we will describe other formalization of the C memory model. 

Norrish (1998) Norrish has formalized a significant fragment of the C89 standard 
using the proof assistant HOL4 . He was the first to describe non-determinism 

and sequence points formally. Our treatment of these features has partly been based 
on his work. Norrish’s formalization of the C type system has some similarities with 
our type system: he has also omitted features that can be desugared and has proven 
type preservation. 
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Contrary to our work, Norrish has used an unstructured memory model based on 
sequences of bytes. Since he has considered the C89 standard in which effective types 
(and similar notions) were not introduced yet, his choice is appropriate. For C99 and 
beyond, a more detailed memory model like ours is needed, see also Section [ 3 ] and 
Defect Report and #451 |26| . 

Another interesting difference is that Norrish represents abstract values (integers, 
pointers and structs) as sequences of bytes instead of mathematical values. Due to 
this, padding bytes retain their value while structs are copied. This is not faithful to 
the C99 standard and beyond. 

Leroy et al. (2006) Leroy et al. have formalized a significant part of C using the Coq 
proof assistant mm- Their part of C, which is called CompCertC, covers most 
major features of C and can be compiled into assembly (PowerPC, ARM and x86) 
using a compiler written in Coq. Their compiler, called CompCert, has been proven 
correct with respect to the CompCert C and assembly semantics. 

The goal of CompCert is essentially different from CH20’s. What can be proven 
with respect to the CompCert semantics does not have to hold for any Cll compiler, 
it just has to hold for the CompCert compiler. CompCert is therefore in its semantics 
allowed to restrict implementation defined behaviors to be very specific (for example, 
it uses 32 bits ints since it targets only 32-bits computing architectures) and allowed 
to give a dehned semantics to various undehned behaviors (such as sequence point 
violations, violations of effective types, and certain uses of dangling pointers). 

The CompCert memory model is used by all languages (from C until assembly) 
of the CompCert compiler mm- The CompCert memory is a finite partial function 
from object identifiers to objects. Each local, global and static variable, and invoca¬ 
tion of malloc is associated with a unique object identifier of a separate object in 
memory. We have used the same approach in CH 2 O, but there are some important 
differences. The paragraphs below discuss the relation of CH 2 O with the first and 
second version of the CompCert memory model. 

Leroy and Blazy (2008) In the first version of the CompCert memory model |41| . 
objects were represented as arrays of type-annotated fragments of base values. Ex¬ 
amples of bytes are thus “the 2nd byte of the short 13” or “the 3rd byte of the pointer 
(o, i)”. Pointers were represented as pairs (o, i) where o is an object identifier and i 
the byte offset into the object o. 

Since bytes are annotated with types and could only be retrieved from memory 
using an expression of matching type, effective types on the level of base types are 
implicitly described. However, this does not match the Cll standard. For example, 
Leroy and Blazy do assign the return value 11 to the following program: 

struct SI ■[ int x; ]■; 

struct S2 ■[ int y; }; 

int f(struct SI *p, struct S2 *q) { 

p->x = 10; 

q->y = 11; 

return p->x; 

} 

int mainO { 

union U ■[ struct SI si; struct S2 s2; } u; 
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printf ("°/,d\ii", f(&u.sl, &u.s2)); 

} 

This code strongly resembles exampl e 1271 6.5.2.3p9] from the Cll standard, 
which is stated to have undehned behavioo GCC and Clang optimize this code to 
print 10, which differs from the value assigned by Leroy and Blazy. 

Apart from assigning too much defined behavior, Leroy and Blazy’s treatment 
of effective types also prohibits any form of “bit twiddling”. 

Leroy and Blazy have introduced the notion of memory injections in [IT]. This 
notion allows one to reason about memory transformations in an elegant way. Our 
notion of memory refinements (Section 17.21) generalize the approach of Leroy and 
Blazy to a tree based memory model. 

Leroy et al. (2012) The second version of CompCert memory model HO] is entirely 
untyped and is extended with permissions. Symbolic bytes are only used for pointer 
values and indeterminate storage, whereas integer and floating point values are rep¬ 
resented as numerical bytes (integers between 0 and 2® — 1). 

We have extended this approach by analogy to bit-representations, representing 
indeterminate storage and pointer values using symbolic bits, and integer values 
using concrete bits. This choice is detailed in Section (021 

As an extension of CompCert, Robert and Leroy have formally proven soundness 
of an alias analysis HO]. Their alias analysis is untyped and operates on the RTL 
intermediate language of CompCert. 

Beringer et al. [7] have developed an extension of CompCert’s memory injections 
to reason about program transformations in the case of separate compilation. The 
issues of separate compilation are orthogonal to those that we consider. 

Appel et al. (201)) The Verified Software Toolchain (VST) by Appel et al. provides 
a higher-order separation logic for Verifiable C, which is a variant of CompCert’s 
intermediate language Clight [H]. 

The VST is intended to be used together with the CompCert compiler. It gives 
very strong guarantees when done so. The soundness proof of the VST in conjunc¬ 
tion with the correctness proof of the CompCert compiler ensure that the proven 
properties also hold for the generated assembly. 

In case the verified program is compiled with a compiler different from CompCert, 
the trust in the program is still increased, but no full guarantees can be given. That 
is caused by the fact that CompCert’s intermediate language Clight uses a specific 
evaluation order and assigns defined behavior to many undefined behaviors of the 
Cll standard. For example, Clight assigns defined behavior to violations of effective 
types and sequence point violations. The VST inherits these defined behaviors from 
CompCert and allows one to use them in proofs. 

Since the VST is linked to CompCert, it uses CompCert’s coarse permission 
system on the level of the operational semantics. Stewart and Appel (3] Chapter 42] 
have introduced a way to use a more fine grained permission system at the level of the 
separation logic without having to modify the Clight operational semantics. Their 
approach shows its merits when used for concurrency, in which case the memory 
model contains ghost data related to the conditions of locks dM]. 

® We have modified the example from the standard slightly in order to trigger optimizations 
by GCC and Clang. 
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Besson et al. (2014) Besson et al. have proposed an extension of the CompCert 
memory model that assigns a defined semantics to operations that rely on the nn- 
merical values of uninitialized memory and pointers [5] . 

Objects in their memory model constitute of lazily evaluated values described by 
symbolic expressions. These symbolic expressions are used to delay the evaluation of 
operations on uninitialized memory and pointer values. Only when a concrete value 
is needed (for example in case of the controlling expression of an if-then-else, for, 
or while statement), the symbolic expression is normalized. Consider: 

int x, *p = &x; 

int y = ((unsigned char*)p) [1] I 1; 

// y has symbolic value " 2 nd pointer byte of p" I 1 

if (y & 1) printf("one\n"); // unique normalization -> OK 

if (y & 2 ) printf("two\n"); // no unique normalization -> bad 

The value of ((unsigned char*)p) [1] I 1 is not evaluated eagerly. Instead, 
the assignment to y stores a symbolic expression denoting this value. During the 
execution of the first if statement, the actual value of y & 1 is needed. In this case, 
y & 1 has the value 1 for any possible numerical value of ((unsigned char*)p) [1] . 
As a result, the string one is printed. 

The semantics of Besson et al. is deterministic by dehnition. Normalization of 
symbolic expressions has dehned behavior if and only if the expression can be normal¬ 
ized to a unique value under any choice of numeral values for pointer representations 
and uninitialized storage. In the second if statement this is not the case. 

The approach of Besson et al. gives a semantics to some programming techniques 
that rely on the numerical representations of pointers and uninitialized memory. For 
example, it gives an appropriate semantics to pointer tagging in which unused bits 
of a pointer representation are used to store additional information. 

However, as already observed by Kang et al. [28], Besson et al. do not give a 
semantics to many other useful cases. For example, printing the object representation 
of a struct, or computing the hash of a pointer value, is inherently non-deterministic. 
The approach of Besson et al. assigns undefined behavior to these use cases. 

The goal of Besson et al. is inherently different from ours. Our goal is to describe 
the Cll standard faithfully whereas Besson et al. focus on de facto versions of C. 
They intentionally assign dehned behavior to many constructs involving uninitialized 
memory that are clearly undehned according to the Cll standard, but that are 
nonetheless faithfully compiled by specihc compilers. 

Ellison and Ro§u (2012) Ellison and Ro§u [T^ITS] have developed an executable 
semantics of the Cll standard using the K-frameworl|3|. Their semantics is very 
comprehensive and describes all features of a freestanding C implementation nn 
4p6] including some parts of the standard library. It furthermore has been thoroughly 
tested against test suites (such as the GCC torture test suite), and has been used as 
an oracle for compiler testing 

Ellison and Ro§u support more C features than we do, but they do not have 
infrastructure for formal proofs, and thus have not established any metatheoretical 
properties about their semantics. Their semantics, despite being written in a formal 
framework, should more be seen as a debugger, a state space search tool, or possibly. 


^ This work has been superseded by Hathhorn et al. m, which is described below. 
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as a model checker. It is unlikely to be of practical use in proof assistants because 
it is defined on top of a large C abstract syntax and uses a rather ad-hoc execution 
state that contains over 90 components. 

Similar to our work, Ellison and Rogu’s goal is to exactly describe the Cll stan¬ 
dard. However, for some programs their semantics is less precise than ours, which 
is mainly caused by their memory model, which is less principled than ours. Their 
memory model is based on CompCert’s: it is essentially a hnite map of objects con¬ 
sisting of unstructured arrays of bytes. 

Hathhorn et al. (2015) Hathhorn et al. [22] have extended the work of Ellison and 
Ro§u to handle more underspecification of Cll. Most importantly, the memory model 
has been extended and support for the type qualifiers const, restrict and volatile 
has been added. 

Hathhorn et al. have extended the original memory model (which was based 
on CompCert’s) with decorations to handle effective types, restrictions on padding 
and the restrict qualifier. Effective types are modeled by a map that associates 
a type to each object. Their approach is less fine-grained than ours and is unable 
to account for active variants of unions. It thus does not assign undefined behavior 
to important violations of effective types and in turn does not allow compilers to 
perform optimizations based on type-based alias analysis. For example: 

// Undefined behavior in case f is called with aliased 
// pointers due to effective types 

int fCshort *p, int *q) { *p = 10; *q = 11; return *p; I 
int mainO { 

union { short x; int y; ]-u = { .y = 0]-; 
return f(&u.x, feu.y); 

} 

The above program has undefined behavior due to a violation of effective types. 
This is captured by our tree based memory model, but Hathhorn et al. require the 
program to return the value 11. When compiled with GCC or Clang with optimiza¬ 
tion level -02, the compiled program returns the value 10. 

Hathhorn et al. handle restrictions on padding bytes in the case of unions, but 
not in the case of structs. For example, the following program returns the value 1 
according to their semantics, whereas it has unspecified behavior according to the 
Cll standard [271 6.2.6.Ip6] (see also Section [H: 

struct S { char a; int b; Is; 

((unsigned char*) (&s)) [1] = 1; 

s.a = 10; // Makes the padding bytes of ’s’ indeterminate 
return ((unsigned char*)(&s))[1]; 

The restrictions on paddings bytes are implicit in our memory model based on 
structured trees, and thus handled correctly. The above examples provide evidence 
that a structured approach, especially combined with metatheoretical results, is more 
reliable than depending on ad-hoc decorations. 

Kang et al. (2015) Kang et al. [TS] have proposed a memory model that gives a 
semantics to pointer to integer casts. Their memory model uses a combination of 
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numerical and symbolic representations of pointer values (whereas CompCert and 
CH 2 O always represent pointer values symbolically). Initially each pointer is repre¬ 
sented symbolically, but whenever the numerical representation of a pointer is needed 
(due to a pointer to integer cast), it is non-deterministically realized. 

The memory model of Kang et al. gives a semantics to pointer to integer casts 
while allowing common compiler optimizations that are invalid in a naive low-level 
memory model. They provide the following motivating example: 


void g(void) { . , 

.. } 

int f (void) ■[ 


int a = 0; 


g(); 


return a; 


} 



In a concrete memory model, there is the possibility that the function g is able 
to guess the numerical representation of &a, and thereby access or even modify a. 
This is undesirable, because it prevents the widely used optimization of constant 
propagation, which optimizes the variable a out. 

In the CompCert and CH 2 O memory model, where pointers are represented 
symbolically, it is guaranteed that f has exclusive control over a. Since &a has not 
been leaked, g can impossibly access a. In the memory model of Kang et al. a pointer 
will only be given a numerical representation when it is cast to an integer. In the 
above code, no such casts appear, and g cannot access a. 

The goal of Kang et al. is to give a unambiguous mathematical model for pointer 
to integer casts, but not necessarily to comply with Cll or existing compilers. Al¬ 
though we think that their model is a reasonable choice, it is unclear whether it is 
faithful to the Cll standard in the context of Defect Report #260 [26]. Consider: 

int x = 0, *p = 0; 
for (uintptr_t i = 0; ; i++) { 
if (i == (uintptr_t)&x) { 
p = (int*)i; break; 

} 

} 

*p = 15; 

printf ("°/,d\n" , x) ; 

Here we loop through the range of integers of type uintptr_t until we have 
found the integer representation i of &x, which we then assign to the pointer p. 

When compiled with gcc -02 (version 4.9.2), the generated assembly no longer 
contains a loop, and the pointers p and q are assumed not to alias. As a result, the 
program prints the old value of x, namely 0. In the memory model of Kang et al. 
the pointer obtained via the cast (int*) i is exactly the same as &x. In their model 
the program thus has defined behavior and is required to print 15. 

We have reported this issue to the GCC bug trackei@. However it unclear whether 
the GCC developers consider this a bug or not. Some developers seem to believe that 
this program has undefined behavior and that GCC’s optimizations are thus justified. 
Note that the cast (intptr_t)&x is already forbidden by the type system of CH 2 O. 
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See https://gcc.gnu.org/bugzilla/show_bug.cgi?id=65752 
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10 Conclusion 

In this paper we have given a formal description of a signihcant part of the non¬ 
concurrent Cll memory model. This formal description has been used in mm as 
part of an an operational, executable and axiomatic semantics of C. On top of this 
formal description, we have provided a comprehensive collection of metatheoretical 
results. All of these results have been formalized using the Coq proof assistant. 

It would be interesting to investigate whether our memory model can be used 
to help the standard committee to improve future versions of the standard. For ex¬ 
ample, whether it could help to improve the standard’s prose description of effective 
types. As indicated on page [3] of Section [TJ the standard’s description is not only 
ambiguous, but also does not cover its intent to enable type-based alias analysis. The 
description of our memory model is unambiguous and allows one to express intended 
consequences formally. We have formally proven soundness of an abstract version of 
type-based alias analysis with respect to our memory model (Theorem 17.21) . 

An obvious direction for future work is to extend the memory model with addi¬ 
tional features. We give an overview of some features of Cll that are absent. 

— Floating point arithmetic. Representations of floating point numbers and the 
behaviors of floating point arithmetic are subject to a considerable amount of 
implementation defined behavior [23 5.2.4.2.2]. 

First of all, one could restrict to IEEE-754 floating point arithmetic, which has 
a clear specification [23] and a comprehensive formalization in Coq [ID]. Soldo 
et al. have taken this approach in the context of CompCert [9] and we see no 
fundamental problems applying it to CH 2 O as well. 

Alternatively, one could consider formalizing all implementation dehned aspects 
of the description of floating arithmetic in the Cll standard. 

— Bitfields. Bithelds are fields of struct types that occupy individual bits 113 
6.7.2.1p9]. We do not foresee fundamental problems adding bitfields to CH 2 O as 
bits already constitute the smallest unit of storage in our memory model. 

— Untyped malloc. CH 2 O supports dynamic memory allocation via an operator 
allocT e close to C-t—l-’s new operator. The alloc,- e operator yields a t* pointer 
to storage for an r-array of length e. This is different from C’s malloc function 
that yields a void* pointer to storage of unknown type [13 7.22.3.4]. 

Dynamic memory allocation via the untyped malloc function is closely related to 
unions and effective types. Only when dynamically allocated storage is actually 
used, it will receive an effective type. We expect one could treat malloced objects 
as unions that range over all possible types that fit. 

— Restrict qualifiers. The restrict qualifier can be applied to any pointer type 
to express that the pointers do not alias. Since the description in the Cll stan¬ 
dard [23 6.7.3.1] is ambiguous (most notably, it is unclear how it interacts with 
nested pointers and data types), formalization and metatheoretical proofs may 
provide prospects for clarification. 

— Volatile qualifiers. The volatile qualifier can be applied to any type to indi¬ 
cate that its value may be changed by an external process. It is meant to prevent 
compilers from optimizing away data accesses or reordering these [23 footnote 
134]. Volatile accesses should thus be considered as a form of I/O. 

— Concurrency and atomics. Shared-memory concurrency and atomic opera¬ 
tions are the main omission from the Cll standard in the CH 2 O semantics. 
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Although shared-memory concurrency is a relatively new addition to the C and 
C-I--I- standards, there is already a large body of ongoing work in this direction, 
see for example |531[51|521|571 ? ]. These works have led to improvements of the 
standard text. 

There are still important open problems in the area of concurrent memory models 
for already small sublanguages of C [3]. Current memory models for these sub¬ 
languages involve just features specific to threads and atomic operations whereas 
we have focused on structs, unions, effective types and indeterminate memory. 
We hope that both directions are largely orthogonal and will eventually merge 
into a fully fledged Cll memory model and semantics. 
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